PERF: Native C++ parameter detection and execute pipeline

[Gaurav Sharma (bewithgaurav)]

Work Item / Issue Reference

AB#44979
AB#49196

GitHub Issue: #500

---

Summary

Moves parameter type detection and binding from Python into a native C++ pipeline using raw CPython API calls. The new DDBCSQLExecute handles type detection → parameter binding → SQLExecute in a single FFI crossing, eliminating per-parameter Python overhead entirely.

What changed:

• DetectParamTypes — C++ type detection using raw CPython API (PyLong_Check, PyDateTime_Check, PyObject_RichCompareBool, etc.) replacing the Python-side _create_parameter_types_list loop for the standard execute path.
• DDBCSQLExecute (formerly DDBCSQLExecuteFast) — single C++ pipeline: detect → bind → execute. ParamInfo never crosses the pybind11 boundary.
• DDBCSQLExecuteLegacy (formerly DDBCSQLExecute) — retained for setinputsizes users only, and annotated in-code as slated for removal once those overrides are handled natively.
• MONEY/SMALLMONEY bounds cached once at init using exact Decimal comparison (not lossy double).
• PyTypeCache (py_type_cache.hpp) stores all type objects as raw PyObject* (not py::object), eliminating pybind11 wrapper overhead on every cache hit.
• NUMERIC mantissa built from four fixed-width uint32 limbs instead of walking every Decimal digit through PyNumber_Multiply/PyNumber_Add. SQL Server caps NUMERIC at 38 digits, so the mantissa always fits 128 bits. Decimal detection measured 2.3–2.9x faster on its own.
• Reference ownership is RAII throughout. ParamInfo::dataPtr holds a py::object rather than a raw PyObject* with a hand-written rule of five, and py_ref.hpp provides steal/borrow so every adoption of a CPython reference states whether it is taking a new reference or borrowing one.
• Parameter detection lives in param_detect.hpp rather than inline in the 6600-line ddbc_bindings.cpp. Header-only because the build is -O3 with no LTO, so a .cpp boundary would also be an inlining boundary.

Routing (cursor.py):

# Standard path (99% of calls): no setinputsizes -> all in C++
# Legacy path: setinputsizes active -> Python type detection + DDBCSQLExecuteLegacy

---

Performance Results 🚀

The Python-side type detection cost was ~2.0–2.3µs per parameter — an isinstance check, ParamInfo object construction, and a pybind11 FFI boundary crossing per parameter, per execute call. The C++ path replaces this with ~35ns/param (raw PyLong_Check + struct field write) — a ~60x faster per-parameter detection.

macOS arm64 (Apple Silicon M-series), Python 3.13

Scenario	After (C++)	Before	Improvement	Python overhead eliminated
3 params	435µs	444µs	2% faster	10µs saved
10 params	321µs	344µs	7% faster	22µs saved
50 params	394µs	502µs	21% faster	107µs saved
100 params	558µs	770µs	28% faster	212µs saved
200 params	644µs	1.08ms	40% faster	432µs saved
500 params	708µs	1.82ms	61% faster	1.12ms saved
1024 params	1.07ms	3.32ms	68% faster	2.25ms saved

Linux aarch64 (Docker container), Python 3.13

Scenario	After (C++)	Before	Improvement	Python overhead eliminated
3 params	158µs	164µs	4% faster	6µs saved
10 params	159µs	178µs	11% faster	19µs saved
50 params	170µs	271µs	37% faster	101µs saved
100 params	209µs	411µs	49% faster	202µs saved
200 params	280µs	651µs	57% faster	371µs saved
500 params	396µs	1.35ms	71% faster	956µs saved
1024 params	746µs	2.75ms	73% faster	2.0ms saved

vs pyodbc (post-PR, macOS)

Params	mssql-python	pyodbc	Gap
3	399µs	388µs	3% gap (near parity)
50	458µs	452µs	1% gap (parity)
200	604µs	550µs	10% gap (down from ~14x pre-PR)
1024	1.31ms	979µs	33% gap (binding overhead, addressable separately)

Customer scenarios (end to end, macOS arm64)

The numbers above isolate driver overhead. These are whole insert workloads, so they also carry the network round trip and SQL Server actually writing the rows, which this PR does not change and which dilutes the percentage. Measured against the merge base (d94debd) with the two builds interleaved across 3 rounds, 7 iterations each, first 2 discarded.

Scenario	Before	After	Speedup
Orders insert (int, varchar, decimal, datetime2)	880.6ms	561.0ms	1.57x
Event log insert (uuid, datetime2, varchar, int)	827.2ms	533.5ms	1.55x
Document insert (nvarchar(max) ~10KB, DAE)	1622.0ms	1021.3ms	1.59x
Wide row insert (50 mixed columns)	1598.6ms	1048.9ms	1.52x
Single-row execute x5000 (4 params)	2103.5ms	2095.9ms	1.00x

Between them these cover every parameter type whose detection moved: int, varchar, decimal, datetime2, uuid, and the nvarchar(max) DAE streaming path.

Single-row execute does not move, and that is the expected result rather than a disappointment. At roughly 420µs per call the cost is the network round trip; detection for 4 parameters was only ever about 9µs of it. The gain scales with parameters per execute, so batched and wide-row work benefits and one-row-at-a-time work stays where it was.

Bottom line

Metric	Value
Avg improvement (50+ params)	~50% faster execute()
Worst-case improvement (1024 params)	73% faster (2ms saved per call)
Per-param overhead reduction	~60x (2.3µs → 35ns)
pyodbc gap closed	From 14x slower (GH-500) to <10% gap at 200 params
Real-world impact	Bulk inserts see 1.5–1.6x throughput end to end (customer scenarios above)

---

Checklist

• Tested locally (macOS arm64 + Linux aarch64)
• Verified perf gain with micro-benchmarks on both platforms, and with end-to-end customer scenarios against the merge base
• CI passing (CodeQL, DevSkim, Black, C++ lint)
• No breaking changes to public API
• setinputsizes users unaffected (routed to legacy path)

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 89%

🎯 Overall Coverage 82%

📈 Total Lines Covered: 7743 out of 9428 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

• mssql_python/cursor.py (100%)
• mssql_python/pybind/ddbc_bindings.cpp (82.3%): Missing lines 247,576-577,718,1821-1822,1965-1966,2060-2061,2064-2067,2079-2087,2110-2111,2141-2143,2164,2181,2197-2200,2210,2629,2744,4430
• mssql_python/pybind/ddbc_bindings.h (100%)
• mssql_python/pybind/param_detect.hpp (92.3%): Missing lines 382-383,389-390,414-415,427-432,465-470,504-505,521-522,538-539,570-571,616-617
• mssql_python/pybind/py_ref.hpp (100%)
• mssql_python/pybind/py_type_cache.hpp (92.7%): Missing lines 48-51

Summary

• Total: 674 lines
• Missing: 70 lines
• Coverage: 89%

mssql_python/pybind/ddbc_bindings.cpp

Lines 243-251

  243 SQLDescribeParamFunc SQLDescribeParam_ptr = nullptr;
  244 
  245 namespace {
  246 
! 247 
  248 const char* GetSqlCTypeAsString(const SQLSMALLINT cType) {
  249     switch (cType) {
  250         STRINGIFY_FOR_CASE(SQL_C_CHAR);
  251         STRINGIFY_FOR_CASE(SQL_C_WCHAR);

---

Lines 572-581

  572                     dataPtr = sqlwcharBuffer->data();
  573                     bufferLength = sqlwcharBuffer->size() * sizeof(SQLWCHAR);
  574                     strLenOrIndPtr = AllocateParamBuffer<SQLLEN>(paramBuffers);
  575                     // Use explicit byte length instead of SQL_NTS so embedded NUL chars
! 576                     // aren't treated as string terminators.
! 577                     *strLenOrIndPtr = static_cast<SQLLEN>(sqlwcharBuffer->size() * sizeof(SQLWCHAR));
  578                 }
  579                 break;
  580             }
  581             case SQL_C_BIT: {

---

Lines 714-722

  714                 dataPtr = static_cast<void*>(sqlTimePtr);
  715                 break;
  716             }
  717             case SQL_C_SS_TIMESTAMPOFFSET: {
! 718                 py::object datetimeType = PyTypeCache::get_datetime_class_obj();
  719                 if (!py::isinstance(param, datetimeType)) {
  720                     ThrowStdException(MakeParamMismatchErrorStr(paramInfo.paramCType, paramIndex));
  721                 }
  722                 // Checking if the object has a timezone

---

Lines 1817-1826

  1817 
  1818 // LEGACY — slated for removal in a future optimization round.
  1819 //
  1820 // Executes the provided query using a ParamInfo list that Python already built,
! 1821 // rather than detecting parameter types in C++. Retained only for setinputsizes()
! 1822 // callers, whose explicit type overrides the native path does not yet honour.
  1823 // Every parameter crosses the pybind11 boundary as a ParamInfo object here, which
  1824 // is the cost SQLExecute_wrap exists to avoid. Once setinputsizes is handled
  1825 // natively this function and its DDBCSQLExecuteLegacy binding both go away.
  1826 //

---

Lines 1961-1970

  1961                     if (matchedInfo->paramCType == SQL_C_WCHAR) {
  1962                         std::u16string utf16 =
  1963                             borrow<py::str>(pyObj).cast<std::u16string>();
  1964                         rc = stream_dae_chunks(
! 1965                             reinterpretU16stringAsSqlWChar(utf16),
! 1966                             utf16.size() * sizeof(SQLWCHAR),
  1967                             putData);
  1968                         if (!SQL_SUCCEEDED(rc)) {
  1969                             LOG("SQLExecute: SQLPutData failed for SQL_C_WCHAR DAE streaming");
  1970                             return rc;

---

Lines 2056-2071

  2056     if (!statementHandle || !statementHandle->get()) {
  2057         return SQL_INVALID_HANDLE;
  2058     }
  2059 
! 2060     SQLHANDLE hStmt = statementHandle->get();
! 2061 
  2062     // Configure forward-only / read-only cursor (matches slow path semantics).
  2063     if (SQLSetStmtAttr_ptr) {
! 2064         SQLSetStmtAttr_ptr(hStmt, SQL_ATTR_CURSOR_TYPE,
! 2065                            (SQLPOINTER)SQL_CURSOR_FORWARD_ONLY, 0);
! 2066         SQLSetStmtAttr_ptr(hStmt, SQL_ATTR_CONCURRENCY,
! 2067                            (SQLPOINTER)SQL_CONCUR_READ_ONLY, 0);
  2068     }
  2069 
  2070     // The encoding-settings dict has the form {"encoding": str, "ctype": int}.
  2071     // Note: the Python layer's SQL_C_CHAR constant is numerically -8, the same

---

Lines 2075-2091

  2075     // real ODBC SQL_CHAR). We default to utf-8 and only honor the dict's
  2076     // encoding when ctype == 1 (real ODBC SQL_CHAR). Otherwise the user's
  2077     // "encoding" value is meant for the wide-char path and we leave it alone.
  2078     std::string charEncoding = "utf-8";
! 2079     if (encoding_settings.contains("ctype") && encoding_settings.contains("encoding")) {
! 2080         int ctype = encoding_settings["ctype"].cast<int>();
! 2081         if (ctype == SQL_C_CHAR /* real ODBC value: 1 */) {
! 2082             charEncoding = encoding_settings["encoding"].cast<std::string>();
! 2083         }
! 2084     }
! 2085 
! 2086     // The cursor.py caller always passes a fresh `list(actual_params)` so this
! 2087     // function is free to mutate slots in place. Even so, every site below uses
  2088     // PyList_SetItem (which decrefs the old slot before stealing the new ref),
  2089     // so the function is safe regardless of who owns the list.
  2090 
  2091     // Run DetectParamTypes BEFORE SQLPrepare so that type-detection errors

---

Lines 2106-2115

  2106             {
  2107                 py::gil_scoped_release release;
  2108                 rc = SQLPrepare_ptr(hStmt, queryPtr, SQL_NTS);
  2109             }
! 2110             if (!SQL_SUCCEEDED(rc)) return rc;
! 2111             statementHandle->clearDescribeCache();
  2112             is_stmt_prepared[0] = py::bool_(true);
  2113         } else {
  2114             ThrowStdException("Cannot execute unprepared statement");
  2115         }

---

Lines 2137-2147

  2137         while (true) {
  2138             {
  2139                 py::gil_scoped_release release;
  2140                 rc = SQLParamData_ptr(hStmt, &paramToken);
! 2141             }
! 2142             if (rc != SQL_NEED_DATA) break;
! 2143 
  2144             const ParamInfo* matchedInfo = nullptr;
  2145             for (auto& info : paramInfos) {
  2146                 if (reinterpret_cast<SQLPOINTER>(const_cast<ParamInfo*>(&info)) == paramToken) {
  2147                     matchedInfo = &info;

---

Lines 2160-2168

  2160 
  2161             if (PyUnicode_Check(pyObj)) {
  2162                 if (matchedInfo->paramCType == SQL_C_WCHAR) {
  2163                     std::u16string u16 =
! 2164                         borrow<py::str>(pyObj).cast<std::u16string>();
  2165                     rc = stream_dae_chunks(
  2166                         reinterpretU16stringAsSqlWChar(u16),
  2167                         u16.size() * sizeof(SQLWCHAR),
  2168                         putData);

---

Lines 2177-2185

  2177                 } else {
  2178                     ThrowStdException("SQLExecute: unsupported C type for str in DAE");
  2179                 }
  2180             } else if (PyBytes_Check(pyObj) || PyByteArray_Check(pyObj)) {
! 2181                 // Handle bytes and bytearray separately — pybind11's bytes
  2182                 // caster does not safely convert bytearray.
  2183                 const char* dataPtr = nullptr;
  2184                 size_t totalBytes = 0;
  2185                 std::string bytesStorage;  // lifetime must span the loop

---

Lines 2193-2204

  2193                     bytesStorage.assign(PyByteArray_AS_STRING(pyObj),
  2194                                         static_cast<size_t>(PyByteArray_GET_SIZE(pyObj)));
  2195                     dataPtr = bytesStorage.data();
  2196                     totalBytes = bytesStorage.size();
! 2197                 }
! 2198 
! 2199                 rc = stream_dae_chunks(dataPtr, totalBytes, putData);
! 2200                 if (!SQL_SUCCEEDED(rc)) return rc;
  2201             } else {
  2202                 ThrowStdException("SQLExecute: DAE only supported for str or bytes");
  2203             }
  2204         }

---

Lines 2206-2214

  2206     }
  2207 
  2208     if (!SQL_SUCCEEDED(rc) && rc != SQL_NO_DATA) return rc;
  2209 
! 2210     // Unbind parameter buffers before they go out of scope.
  2211     // Not called on error paths — diagnostics must remain readable.
  2212     SQLRETURN exec_rc = rc;
  2213     SQLFreeStmt_ptr(hStmt, SQL_RESET_PARAMS);
  2214     return exec_rc;

---

Lines 2625-2633

  2625                     DateTimeOffset* dtoArray =
  2626                         AllocateParamBufferArray<DateTimeOffset>(tempBuffers, paramSetSize);
  2627                     strLenOrIndArray = AllocateParamBufferArray<SQLLEN>(tempBuffers, paramSetSize);
  2628 
! 2629                     py::object datetimeType = PyTypeCache::get_datetime_class_obj();
  2630 
  2631                     for (size_t i = 0; i < paramSetSize; ++i) {
  2632                         const py::handle& param = columnValues[i];

---

Lines 2740-2748

  2740 
  2741                     // Get cached UUID class from module-level helper
  2742                     // This avoids static object destruction issues during
  2743                     // Python finalization
! 2744                     py::object uuid_class = PyTypeCache::get_uuid_class_obj();
  2745                     // Get cached UUID class
  2746 
  2747                     for (size_t i = 0; i < paramSetSize; ++i) {
  2748                         const py::handle& element = columnValues[i];

---

Lines 4426-4434

  4426                     break;
  4427                 }
  4428                 case SQL_TYPE_DATE: {
  4429                     PyObject* dateObj =
! 4430                         PyTypeCache::get_date_class_obj()(buffers.dateBuffers[col - 1][i].year,
  4431                                                             buffers.dateBuffers[col - 1][i].month,
  4432                                                             buffers.dateBuffers[col - 1][i].day)
  4433                             .release()
  4434                             .ptr();

---

mssql_python/pybind/param_detect.hpp

Lines 378-387

  378             // so calling the same method is what keeps the two paths in agreement.
  379             py::object time_obj = steal(PyObject_CallMethod(obj, "isoformat", "s", "microseconds"));
  380             if (!time_obj) throw py::error_already_set();
  381             if (!PyUnicode_Check(time_obj.ptr())) {
! 382                 throw py::type_error("datetime.time.isoformat() must return a str");
! 383             }
  384             Py_ssize_t time_len = PyUnicode_GET_LENGTH(time_obj.ptr());
  385             info.columnSize = std::max<SQLULEN>(info.columnSize, time_len);
  386             // PyList_SetItem (lowercase) decrefs the old slot before stealing the new
  387             // reference; safe here because cursor.py already passed a fresh list copy.

---

Lines 385-394

  385             info.columnSize = std::max<SQLULEN>(info.columnSize, time_len);
  386             // PyList_SetItem (lowercase) decrefs the old slot before stealing the new
  387             // reference; safe here because cursor.py already passed a fresh list copy.
  388             if (PyList_SetItem(params, i, time_obj.release().ptr()) != 0) {
! 389                 throw py::error_already_set();
! 390             }
  391             continue;
  392         }
  393 
  394         // --- Decimal ---

---

Lines 410-419

  410             py::object digits_obj = steal(PyObject_GetAttrString(as_tuple_ptr.ptr(), "digits"));
  411             if (!digits_obj) throw py::error_already_set();
  412 
  413             if (!PyTuple_Check(digits_obj.ptr())) {
! 414                 throw py::type_error("Decimal.as_tuple().digits must be a tuple");
! 415             }
  416 
  417             Py_ssize_t num_digits = PyTuple_GET_SIZE(digits_obj.ptr());
  418 
  419             // Read the exponent at full width and range-check it BEFORE narrowing to int.

---

Lines 423-436

  423             // any precision, so treat overflow as precision overflow rather than propagating
  424             // OverflowError, matching what the legacy Python path reports.
  425             long long exponent_ll = PyLong_AsLongLong(exponent_obj.ptr());
  426             if (exponent_ll == -1 && PyErr_Occurred()) {
! 427                 PyErr_Clear();
! 428                 throw py::value_error(
! 429                     "Precision of the numeric value is too high. "
! 430                     "The maximum precision supported by SQL Server is " +
! 431                     std::to_string(MAX_NUMERIC_PRECISION) + ".");
! 432             }
  433             // Bound before any arithmetic or negation. MAX_NUMERIC_PRECISION on both sides is
  434             // wider than anything bindable, and keeps -exponent well clear of INT_MIN, whose
  435             // negation would be signed-overflow UB.
  436             if (exponent_ll > MAX_NUMERIC_PRECISION || exponent_ll < -MAX_NUMERIC_PRECISION) {

---

Lines 461-474

  461             else
  462                 precision = -exponent;
  463 
  464             if (precision > MAX_NUMERIC_PRECISION) {
! 465                 throw py::value_error(
! 466                     "Precision of the numeric value is too high. "
! 467                     "The maximum precision supported by SQL Server is " +
! 468                     std::to_string(MAX_NUMERIC_PRECISION) + ", but got " +
! 469                     std::to_string(precision) + ".");
! 470             }
  471 
  472             // Check SMALLMONEY first, then widen to MONEY, so common small values keep the narrowest
  473             // exact range while still accepting larger fixed-point values supported by SQL Server.
  474             // MONEY/SMALLMONEY: SQL Server stores these as fixed-point integers internally.

---

Lines 500-509

  500                 PyObject* raw = formatted.release().ptr();
  501                 if (PyList_SetItem(params, i, raw) != 0) {
  502                     // PyList_SetItem steals (decrefs) the item even on failure,
  503                     // so raw is already freed — do NOT Py_DECREF here.
! 504                     throw py::error_already_set();
! 505                 }
  506                 continue;
  507             }
  508 
  509             // Build SQL_NUMERIC_STRUCT from the Decimal object. Store as a pybind11-castable

---

Lines 517-526

  517             py::object numeric_obj = py::cast(nd);
  518             PyObject* raw = numeric_obj.release().ptr();
  519             if (PyList_SetItem(params, i, raw) != 0) {
  520                 // PyList_SetItem steals (decrefs) the item even on failure.
! 521                 throw py::error_already_set();
! 522             }
  523             continue;
  524         }
  525 
  526         // --- UUID ---

---

Lines 534-543

  534             info.columnSize = 16;
  535             info.decimalDigits = 0;
  536             if (PyList_SetItem(params, i, bytes_le) != 0) {
  537                 // PyList_SetItem steals (decrefs) the item even on failure.
! 538                 throw py::error_already_set();
! 539             }
  540             continue;
  541         }
  542 
  543         // --- Unknown type: raise TypeError (matches Python _map_sql_type) ---

---

Lines 566-575

  566     int sign_val = static_cast<int>(PyLong_AsLong(sign_obj.ptr()));
  567     if (sign_val == -1 && PyErr_Occurred()) throw py::error_already_set();
  568 
  569     if (!PyTuple_Check(digits)) {
! 570         throw py::type_error("Decimal.as_tuple().digits must be a tuple");
! 571     }
  572 
  573     // SQL Server precision counts all stored decimal digits, while scale is just the
  574     // fractional digits. A positive exponent moves trailing zeros into the integer part;
  575     // a negative exponent means scale = -exponent and precision must still cover leading

---

Lines 612-621

  612     for (int j = 0; j < exponent; ++j) {
  613         overflow |= mul10_add(0);
  614     }
  615     if (overflow != 0) {
! 616         throw py::value_error("Decimal magnitude exceeds the 16-byte SQL NUMERIC capacity");
! 617     }
  618 
  619     NumericData nd;
  620     nd.precision = static_cast<SQLCHAR>(precision);
  621     nd.scale = static_cast<SQLSCHAR>(scale);

---

mssql_python/pybind/py_type_cache.hpp

Lines 44-55

  44 // type detection in Python and can therefore reach here without the cache being warm;
  45 // it can be dropped once that path is removed.
  46 inline PyObject* get_cached_class(PyObject* cached, const char* module_name, const char* attr_name) {
  47     if (cache_initialized && cached) return cached;
! 48     py::object mod = steal(PyImport_ImportModule(module_name));
! 49     if (!mod) return nullptr;
! 50     return PyObject_GetAttrString(mod.ptr(), attr_name);
! 51 }
  52 
  53 // One-time init. Uses local py::objects so exception cleanup is automatic;
  54 // only .release() into globals after ALL acquisitions succeed.
  55 inline void initialize() {

---

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 61.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 75.4%
mssql_python.__init__.py: 77.6%
mssql_python.row.py: 77.6%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 81.4%
mssql_python.pybind.connection.connection.cpp: 84.3%
mssql_python.logging.py: 85.5%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[Copilot]

Pull request overview

This PR shifts the primary cursor.execute() pipeline from Python into native C++ by adding a raw-CPython DetectParamTypes fast path and routing calls to a single DDBCSQLExecute FFI entrypoint when setinputsizes isn’t active, targeting large parameter-count performance regressions (GH-500).

Changes:

• Added a native DetectParamTypes → BindParameters → SQLExecute pipeline (DDBCSQLExecute) to avoid per-parameter Python/pybind11 overhead.
• Preserved a legacy path (DDBCSQLExecuteLegacy) for setinputsizes users and updated Python routing accordingly.
• Added parity and regression tests covering fast/slow path equivalence, subclass handling, DAE streaming, and refcount safety.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
tests/test_023_fast_path_parity.py	Adds fast-vs-legacy parity tests and regression coverage for the new native execute pipeline.
tests/test_010_pybind_functions.py	Updates exposed-function expectations to include DDBCSQLExecuteLegacy.
mssql_python/pybind/ddbc_bindings.cpp	Implements native type detection, new execute entrypoints, and various binding/DAE handling updates.
mssql_python/cursor.py	Routes execute() to DDBCSQLExecute for the primary path and to DDBCSQLExecuteLegacy when setinputsizes overrides are present.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Jahnvi Thakkar (jahnvi480)]

Blocking: static_cast<int>(PyLong_AsLong(...)) narrows the exponent to 32 bits before we validate it. On LP64 platforms (Linux/macOS, where long is 64-bit), Decimal("1E+4294967297") has exponent 4294967297, which truncates to 1, sails past the precision <= 38 gate, and silently binds 10 — while the legacy Python path (arbitrary-precision int) computes precision > 38 and raises. (On 64-bit Windows long is 32-bit, so PyLong_AsLong overflows and line 416 catches it instead — but the value still can't bind, and it's a platform-dependent divergence either way.) INT_MIN-style truncations also invite UB on the later negation, and huge positive results blow up the packing loop.

Please reject num_digits > 38 and exponents outside [-38, 38] (treating a PyLong_AsLong overflow — returns -1 with PyErr_Occurred() — as precision overflow) before narrowing/arithmetic. Worth tests around INT_MIN, INT_MAX, and 2**32.

[Jahnvi Thakkar (jahnvi480)]

The geometry override is gated behind !info.isDAE, so a WKT string over 4000 UTF-16 units never gets the geometry treatment — it stays on the generic long-string DAE path (SQL_VARCHAR, columnSize 0, DAE=true). The legacy _map_sql_type checks the geometry prefix first and returns SQL_WVARCHAR / SQL_C_WCHAR / len(param) / non-DAE regardless of length. Large polygons are realistic, so this changes both the SQL type and the execution path.

Please move geometry detection ahead of the length/DAE decision (and across all Unicode storage kinds, not just 1-byte) so both paths agree, then add a >4000-char POLYGON parity test.

[Jahnvi Thakkar (jahnvi480)]

This hand-formats HH:MM:SS.ffffff from the raw time fields, which drops any tzinfo and bypasses subclass isoformat overrides. The legacy path calls value.isoformat(timespec="microseconds") (see _normalize_time_param in cursor.py), so an aware datetime.time produces e.g. 01:02:03.000004+05:30 there but 01:02:03.000004 here.

Simplest fix: call isoformat("microseconds") from C++ and bind the returned string/length instead of building it manually.

[Jahnvi Thakkar (jahnvi480)]

On Windows PARAM_C_TYPE_TEXT resolves to the real ODBC SQL_C_CHAR (1), but Python's SQL_C_CHAR constant is -8 (identical to SQL_C_WCHAR). So the legacy path always binds text as wide, while the native path binds ASCII strings / money-text / time as narrow on Windows — a genuine C-type and encoding-path divergence. CI is green, but for exact parity I'd bind SQL_C_WCHAR on every platform.

If narrow Windows binding is intentional, let's state it as the parity contract and test it explicitly (incl. non-ASCII collation).

[Jahnvi Thakkar (jahnvi480)]

This "slow path" helper calls setinputsizes(...), which forces the supplied SQL type and bypasses _map_sql_type entirely. So the suite is really comparing native detection against hand-specified types — it can't catch the geometry / aware-time / Windows C-type / column-size / DAE divergences that actually matter (and the file comment claiming it exercises _map_sql_type is inaccurate).

Suggest exposing a test-only native detector and asserting all five fields (SQL type, C type, column size, decimal digits, DAE) directly against _map_sql_type, using fresh parameter lists and the threshold / aware-time / Decimal-exponent edge cases.

[Jahnvi Thakkar (jahnvi480)]

Rejecting non-finite Decimals as ValueError here is reasonable, but the legacy path raises different types — NaN raises decimal.InvalidOperation (from the money <= comparison) and Infinity reaches _get_numeric_data and raises TypeError. If we want exception parity, add the same explicit non-finite check on the Python side so both raise ValueError, and assert the exact type.

[Jahnvi Thakkar (jahnvi480)]

If BindParameters binds some parameters and then throws, we unwind without SQL_RESET_PARAMS, so the HSTMT keeps pointers into paramBuffers that are about to be destroyed, until the next reset. Exposure is limited (the next execute resets first), but an exception-only scope guard that resets params once binding has begun would make the invariant safe. Keep the existing ODBC-return error path untouched so diagnostics stay readable.

[Jahnvi Thakkar (jahnvi480)]

Leftover old name — SQLExecuteFast: here (and again at ~2134 and ~2158, plus a comment near line 1999). The function was renamed away from "fast"; these strings should follow so logs/messages aren't misleading.

[Jahnvi Thakkar (jahnvi480)]

For immutable bytes this copies the whole payload into bytesStorage before streaming. matchedInfo->dataPtr holds a strong reference for the duration, so you can stream straight from PyBytes_AS_STRING(pyObj) / PyBytes_GET_SIZE(pyObj) and skip the copy. (The bytearray branch still needs its copy since it's mutable.)

[Jahnvi Thakkar (jahnvi480)]

This linearly scans every param for each DAE token. Since the token is a stable ParamInfo* (never reallocated — the vector is sized up front), you can reinterpret_cast it back directly (or pass an index) instead of the O(N) search per chunk.