Skip to content

Support subtraction and ordering between converted DateTime values and datetime.date - #143

Open
jhonabreul wants to merge 2 commits into
QuantConnect:masterfrom
jhonabreul:feature-datetime-date-coercion
Open

Support subtraction and ordering between converted DateTime values and datetime.date#143
jhonabreul wants to merge 2 commits into
QuantConnect:masterfrom
jhonabreul:feature-datetime-date-coercion

Conversation

@jhonabreul

@jhonabreul jhonabreul commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

What does this implement/fix? Explain your changes.

Python algorithms mixing converted .NET DateTime values (self.time, contract.id.date, option/future expiries) with pure Python datetime.date objects fail at runtime:

TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'datetime.date'
TypeError: can't compare datetime.datetime to datetime.date

The typical shape is DTE math, e.g. dte = (contract.id.date - self.time.date()).days, the single most repeated runtime arithmetic failure across user deployments.

Cause: Converter.ToPython emits stock datetime.datetime instances, and CPython refuses subtraction and ordering between datetime and pure date operands.

The fix:

  • Converter now emits System.DateTime values as instances of a datetime.datetime subclass, defined once (lazily) in a synthetic clr_datetime module.
  • The subclass's __sub__/__rsub__/__lt__/__le__/__gt__/__ge__ coerce against pure datetime.date operands using the datetime's .date() part; any other operand defers to the stock implementation. Since datetime subclasses date, reflected-operator precedence makes both operand orders work (converted - date and date - converted).
  • Equality and hashing are deliberately untouched: datetime == date stays False, since making it true would break the hash contract for dicts/sets.
  • __repr__ renders identically to stock (datetime.datetime(...)), and __reduce_ex__ pickles instances as plain datetime.datetime, so pickled payloads and deep copies never reference the synthetic module.
  • tests/test_conversion.py test_datetime_conversion relaxed its exact-type assertion (type(...) is datetime) to isinstance — deliberate, the emitted type is now a subclass.

Does this close any currently open issues?

No open pythonnet issue; addresses the datetime.datetime vs datetime.date failure class quantified in QuantConnect/Agents#305 (14/14 affected deployments in the sampled fleet).

Any other comments?

  • Includes the version bump to 2.0.65 as its own commit (<Version>, AssemblyVersion/AssemblyFileVersion, and the perf-test baseline package references). Merging publishes 2.0.65 to NuGet; downstream bumps are staged in Improve error messages for datetime vs date operations in Python algorithms Lean#9659 and QuantConnect/LeanCloud.Services#253 (both blocked on indexing).

  • Perf (existing ConvertDateTimeRoundTripPerformance, 500k ToPython+ToManaged round trips, Release): Unspecified kind 521ms before → 515ms after; Utc kind 1542ms → 1231ms. No regression.

  • New embed tests (TestConverter.ConvertedDateTime*, 3 of 5 red before the fix): subtraction with a date in both operand orders, ordering in both orders, same-day comparison semantics, plain-datetime behavior parity (equality, hash, arithmetic with datetime/timedelta, str/repr/strftime), and pickle degradation to plain datetime.

  • Python-side tests/test_conversion.py::test_datetime_date_coercion exercises the same through a real CLR boundary (ConversionTest.DateTimeField).

  • End-to-end against Lean (master + Improve error messages for datetime vs date operations in Python algorithms Lean#9659, this branch's Python.Runtime.dll injected over the test output, mirroring the Lean Python CI workflows): Python unit tests 16045 passed / 0 failed / 9 skipped; Python regression algorithms 324/324 passed.

  • Full embed suite: 978 passed, 0 failed, 8 skipped. Full Python test suite: 457 passed, 1 pre-existing environment-related failure (test_explicit_assembly_load, fails identically on master).

  • Operations where both operands are pure Python objects (e.g. datetime.now() - date.today()) still raise, as pythonnet is not involved; a companion Lean PR improves those error messages: Improve error messages for datetime vs date operations in Python algorithms Lean#9659. No merge-order dependency between the two.

Checklist

Check all those that are applicable and complete.

  • Make sure to include one or more tests for your change
  • If an enhancement PR, please create docs and at best an example
  • Ensure you have signed the .NET Foundation CLA
  • Add yourself to AUTHORS
  • Updated the CHANGELOG

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants