Skip to content

fix(gsm): follow-up review fixes + restore the connections/gsm package - #615

Merged
yozik04 merged 9 commits into
devfrom
fix/gsm-followup-review
Aug 14, 2026
Merged

fix(gsm): follow-up review fixes + restore the connections/gsm package#615
yozik04 merged 9 commits into
devfrom
fix/gsm-followup-review

Conversation

@yozik04

@yozik04 yozik04 commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Follow-up to #613 and #614, from the post-merge critical review.

⚠️ This also restores #614

#614's content never reached dev. It was stacked on fix/611-gsm-line-framing and GitHub marked it MERGED when it merged into that branch — but #613 was merged to dev beforehand, so paradox/connections/gsm/ does not exist on dev today. The first commit here is a cherry-pick of 9803043 to bring it back.

git merge-base --is-ancestor 9803043 origin/dev   # false

Fixes

1. send_message was async but awaited nothing — the root cause of item 7

ConnectionProtocol.send_message is a synchronous abstract method. GsmSerialProtocol overrode it as async def around a bare transport.write(). That LSP violation is why it collided with serial.protocol.SerialConnectionProtocol in the first place: #613 renamed the class to dodge the collision, and #614 then had to override Connection.write() to raise NotImplementedError so nobody could reach the un-awaited coroutine.

Making it synchronous deletes both workarounds. send_command() now uses the inherited Connection.write(), and the missing check_active() guard is in place.

2. LineFramer emitted the tail of an over-long line as a valid frame

Before, after discarding an overrun the framer would treat whatever followed up to the next terminator as a complete line:

f = LineFramer(terminator=b"\r\n", max_line_length=8)
f.feed(b"X" * 12)          # [] - discarded, good
f.feed(b"TAIL\r\nOK\r\n")  # [b"TAIL", b"OK"]  <- b"TAIL" is corrupt

It now resynchronises: bytes up to and including the next terminator are dropped, so only b"OK" comes through. reset() clears the resync state.

3. drop_blank_linesdrop_whitespace_lines

The old name conflated two things. Empty payloads are now always skipped; skipping whitespace-only lines is the opt-in behaviour the name now describes. PRT3 is the only caller and keeps its existing semantics.

4. connect() did not actually bound the port open

open_timeout was scheduled with call_later, but it only resolves connected_future — useless if create_serial_connection is what hangs, since connect() is not awaiting that future yet. The open is now wrapped in asyncio.wait_for.

Tests

1633 passing. New coverage for the overflow resync, reset() clearing it, the bounded open, and write() reaching the transport (replacing the NotImplementedError test).

5. GSM outbound SMS never worked at all

Not a regression — the code at d347e0b, before any of these PRs, has all three of these defects verbatim. The test for it was left commented out in tests/interfaces/test_gsm.py:

# level = EventLevel.INFO
# await interface.send_message("bla", level)

a. send_message was async over a synchronous base. GSMTextInterface was the codebase's only async override of AbstractTextInterface.send_message, which handle_notify and handle_panel_event call directly. Every notification built a coroutine that nobody awaited and dropped it — the source of the RuntimeWarning: coroutine 'GSMTextInterface.send_message' was never awaited seen throughout the test suite. It is now synchronous and schedules the send as a task.

b. The response queue was starved after init. connect() finishes with set_recv_callback(self.data_received), after which on_message routed every line to the callback and none to the queue — so send_command()'s queue.get() could never be satisfied. Even a correctly awaited SMS could only time out.

set_recv_callback is typed Callable[[bytes], bool], but the result was discarded. Honouring it gives URC demultiplexing for free:

if self.recv_callback is not None and self.recv_callback(message):
    return
self.queue.put_nowait(message)

data_received now returns True only for lines it consumed (+CMT and its body, +CUSD); OK, ERROR and +CMGS: fall through to the waiting command.

c. AT+CMGS was sent as one blob. Text-mode SMS is a two-stage exchange: the modem answers with a > entry prompt and only then accepts the body, terminated by Ctrl-Z rather than CRLF. Sending both at once leaves the modem sitting in entry mode with nothing sent.

The prompt is \r\n> unterminated, so LineFramer can never surface it. GsmSerialProtocol gains a one-shot expect_prompt() that a caller arms when it knows a prompt is due; unarmed, > is correctly treated as a line that has not finished arriving. Failure paths send ESC so the modem does not swallow the next command as message text.

Tests

1654 passing, up from 1633 on dev. Each fix was verified to be genuinely covered by reverting it and confirming the new tests fail:

Reverted Failing tests
callback return value ignored 5, incl. test_unsolicited_lines_do_not_satisfy_a_pending_command
prompt detection removed 5, incl. test_sms_is_a_two_stage_exchange

test_sms_is_a_two_stage_exchange drives real bytes through the real protocol and asserts the full command → prompt → body → +CMGS:/OK sequence, which is what the commented-out test was reaching for.


Follow-up: rubber-duck review of the above

Reviewing the SMS work turned up five more defects. Three were confirmed by running them, not just by reading.

6. The command channel had no mutual exclusion (blocking)

Two notifications arriving together each schedule a send. Observed: both commands reach the modem before either reply comes back.

>>> writes before any reply: [b'AT+A\r\n', b'AT+B\r\n']

For AT+CMGS that is worse than mispaired replies — between the > prompt and the Ctrl-Z the modem treats everything written as message text, so the second command becomes the body of the first SMS.

The lock lives in GSMTextInterface rather than the transport: an SMS exchange spans send_command + write_raw + read, so guarding it inside the connection would require a reentrant lock. It is also held across the whole init sequence, so a half-initialised modem cannot see an SMS interleaved between ATE0 and AT+CMGF=1.

7. A stale reply satisfied the next command (blocking)

The late answer to a timed-out command sat in the queue and was handed to whatever came next:

>>> AT+B received: b'LATE-REPLY-TO-A'

send_command now drains before writing — anything already queued predates the command. clear() drains in place instead of replacing the queue object, which previously stranded any coroutine already blocked in read().

8. expect_prompt() stayed armed after a timeout

Nothing disarmed it on the error paths, leaving the next unterminated line primed to be read as an SMS prompt. Now cleared in a finally.

9. A modem drop was fatal until restart

run() looped while not self.modem_connected and returned once connected. on_connection_loss clears the transport's flag but nothing ever reset the interface's, so an unplugged or reset modem meant no more alerts until PAI restarted — on an alarm interface, silently. run() now watches the transport and rebuilds the port, and connect() closes the previous one first (a failed retry used to leak the last attempt).

10. The init helper was dead code around a real bug

while r != expected:
    r = await self.port.send_command(message)
    data = b""
    if r == b"ERROR": raise ...
    while r != expected:
        r = await self.port.read()
        data += r + b"\n"

The outer loop can never iterate twice (the inner one only exits when r == expected), data is accumulated and thrown away, and an ERROR arriving in the inner loop is ignored — so a rejected init command cost a 5 s timeout instead of reporting itself. Replaced by _at_command(), which shares the final-result reader with the SMS path.

Also: in-flight send tasks are now tracked, so stop() cancels them and the event loop keeps a strong reference (asyncio only holds a weak one).

Tests

1664 passing, up from 1633 on dev. Every fix was verified by reverting it and confirming the new tests fail:

Reverted Failing test
command lock test_concurrent_sends_are_serialised
queue drain test_a_stale_reply_does_not_satisfy_the_next_command
prompt disarm test_a_timed_out_command_disarms_the_prompt
reconnect check test_run_reconnects_after_the_modem_drops
callback return value 5 tests, incl. test_unsolicited_lines_do_not_satisfy_a_pending_command
prompt detection 5 tests, incl. test_sms_is_a_two_stage_exchange

Closes item 8 of #611. SerialCommunication was a parallel hand-rolled
reimplementation of the Connection abstraction living in interfaces/, and
it shared its name with connections/serial/connection.py's panel transport
-- the same non-substitutable collision item 7 fixed for the protocols.

- New paradox/connections/gsm/ package holding GsmSerialProtocol and
  GsmSerialConnection, mirroring ip/, serial/ and prt3/.
- GsmSerialConnection extends Connection for lifecycle only: the
  protocol-aware connected property, _protocol plumbing, and close(), which
  the GSM transport had no equivalent of. on_message stays overridden to
  keep the queue and recv_callback dispatch, because AT command/response
  does not fit Connection's parsed-message handler registry.
- The request/response method is renamed send_command(). Keeping it as
  write() would have collided with Connection.write(), which is a
  synchronous fire-and-forget and would leave send_message's coroutine
  un-awaited.
- GSMTextInterface.stop() now closes the port instead of logging a TODO.

Bugs fixed in the moved code, all of which connections/serial/connection.py
already got right:

- on_connection and on_connection_loss set connected_future unguarded, so a
  modem drop after a successful connect raised InvalidStateError.
- The open timeout handle was never cancelled, and the constructor's timeout
  argument was accepted and then ignored in favour of a hardcoded 5.
- send_command's timeout argument was ignored in favour of a hardcoded 5.
- The port check was os.path.exists, which misses the far more common
  permission failure that os.access(R_OK|W_OK) catches.
- connect() logged "Could not connect" via logger.exception outside any
  except block, emitting a bogus "NoneType: None" traceback.
…flow

Follow-up review fixes on top of #613 and #614.

- GsmSerialProtocol.send_message was declared async but awaited nothing,
  violating the synchronous ConnectionProtocol.send_message contract. That
  LSP violation is what forced the class rename and the Connection.write()
  NotImplementedError override; making it synchronous removes both
  workarounds and adds the missing check_active().
- LineFramer emitted the tail of an over-long line as if it were a whole
  line. It now resynchronises on the next terminator after a discard.
- LineFramer's drop_blank_lines is renamed to drop_whitespace_lines: empty
  payloads are always skipped, whitespace-only ones are opt-in.
- GsmSerialConnection.connect() bounds the port open itself with
  asyncio.wait_for; the existing call_later handler could not fire while
  create_serial_connection was the thing hanging.
Outbound SMS has never worked, before or after the framing rewrite: the
code at d347e0b has all three of these defects verbatim, and the test for
it was left commented out.

- GSMTextInterface.send_message was the codebase's only async override of
  the synchronous AbstractTextInterface.send_message. handle_notify and
  handle_panel_event call it directly, so every notification built a
  coroutine that nobody awaited and silently discarded it. It is now
  synchronous and schedules the send as a task.

- connect() ends by installing data_received as the recv callback, after
  which on_message routed *every* line to it and none to the queue. That
  starved send_command for the life of the process, so even a correctly
  awaited SMS could only time out. set_recv_callback is typed
  Callable[[bytes], bool]; that contract is now honoured, and
  data_received reports True only for lines it consumed (+CMT and its
  body, +CUSD). Command replies fall through to the queue.

- AT+CMGS was sent as a single blob. Text-mode SMS is a two-stage
  exchange: the modem answers with a '> ' entry prompt and only then
  accepts the body, terminated by Ctrl-Z rather than CRLF. The prompt
  carries no terminator, so LineFramer can never surface it; the protocol
  gains a one-shot expect_prompt() that a caller arms when it knows a
  prompt is due, which avoids mistaking an unfinished line for one.
  Failure paths send ESC so the modem does not swallow the next command
  as message text.
Five defects found reviewing the SMS work, three verified by execution.

- The command channel had no mutual exclusion. Two notifications arriving
  together put two AT commands on the wire before either reply came back;
  for AT+CMGS that means the second command is swallowed as the body text
  of the first SMS. Exchanges are now serialised by a lock in the
  interface, which owns the AT semantics -- an SMS spans send_command,
  write_raw and read, so the transport cannot guard it without becoming
  reentrant. The lock is also held across the whole init sequence.

- A reply arriving after its command timed out stayed in the queue and
  satisfied the *next* command. send_command now drains the queue before
  writing: anything already there predates the command. clear() drains in
  place rather than replacing the queue, which used to strand a coroutine
  already blocked in read().

- expect_prompt() stayed armed when its command timed out, priming the
  next unterminated line to be read as an SMS prompt. send_command now
  disarms it in a finally.

- run() exited once connected and nothing ever reset modem_connected, so
  a modem reset or unplug was fatal until PAI restarted. It now watches
  the transport's connected flag and rebuilds the port. connect() closes
  the previous port first, which also stops a failed retry leaking the
  last attempt.

- The init helper re-sent the command in an outer loop that could never
  iterate twice, accumulated a `data` string it discarded, and ignored an
  ERROR arriving in its inner loop -- so a rejected init command cost a
  five second timeout instead of reporting itself. Replaced by
  _at_command(), sharing the final-result reader with the SMS path.

In-flight send tasks are now tracked, so stop() cancels them and the loop
keeps a strong reference to them.
No behaviour change; the existing tests cover all of it.

- _read_final_result computed its own deadline. asyncio.wait_for around
  an inner loop does the same thing without the arithmetic.

- Its first_line parameter existed only because _at_command went through
  send_command, which pops the first line and then has to hand it back,
  forcing a None guard on every iteration. _at_command now writes the
  command itself, so both the parameter and the guard go away.
  send_command remains for the one case that needs it: the SMS prompt.

- Dropped the _OK constant, which was indirection for a two-byte literal,
  and folded the bare ERROR check into the error prefix tuple.

- AT_COMMAND_TIMEOUT duplicated DEFAULT_COMMAND_TIMEOUT in the connection
  module. Import the one that already exists.

- run() checked `self.port is not None`, which cannot be observed: stop()
  has no await between cancelling the task and clearing the port.
send_command had grown into a 25 line wrapper with two optional flags and
exactly one caller. It did five things -- connection check, drain, log,
arm the prompt, write, read -- none of which belong together.

Deleting it leaves the transport with four orthogonal primitives:
clear(), write(), write_raw() and read(). The prompt expectation moves
onto read(), which is what it actually affects, and both exchanges become
the same three steps: clear, write, read.

Also removes an unnecessary list() around the send task set, flagged by
SonarCloud. cancel() does not run done callbacks synchronously, so the
set cannot mutate while it is iterated.

The other seven SonarCloud findings ask for asyncio.timeout() in place of
asyncio.wait_for(). That context manager is 3.11+ and this project
supports 3.8, so they do not apply.

The stale-reply test moves to the interface, where the drain now lives,
and is split into the two cases that actually matter: a stale OK
satisfying the next command, and a stale line being read as the SMS
prompt. The single test it replaced passed with the fix reverted --
_read_final_result skips unrecognised lines, so stale junk was never the
risk.
@sonarqubecloud

Copy link
Copy Markdown

@yozik04
yozik04 merged commit 0e4ba27 into dev Aug 14, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant