Fix the AirTouch 2+ connection dropout: ACK address (0xC0) + keep-alive#18
Fix the AirTouch 2+ connection dropout: ACK address (0xC0) + keep-alive#18aramshaw wants to merge 3 commits into
Conversation
Add 4 ControlStatusMessage subtypes identified by aramshaw, which all require an acknowledgement, otherwise the server will start ignoring the client.
The ACK's top-level address byte must carry the control/status identifier (0xC0), not the usual client address (0x80). Verified on a live AirTouch 2+: acking the status broadcasts with 0x80 triggers a continuous spam storm (~1.7 broadcasts/sec, ~100x normal) until the session is unusable; 0xC0 is accepted and the controller stays calm. It looks redundant with the message-type byte but the controller genuinely validates it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Acking the broadcasts keeps the controller happy while it is talking, but it still drops the TCP session after ~16 min of silence (it stops broadcasting when nothing changes, e.g. overnight) - the recurring "connection lost" / unavailable problem. A lightweight AC-status request every 4 min keeps the session alive. Verified: 3 days with zero dropouts (vs several per day before). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Addendum: Master's TCP keepalive doesn't prevent dropouts (I tested this), the poll is what does. I ran a local soak test with the keepalive removed but have not touched it here. |
|
Oh this is really interesting, not sure if there's an easy way for me to drop in your changes @aramshaw on my live system but I'd love to fix this once and for all. I've been back using a automation keepalive for a while. For me I found the TCP keepalives helped a bit, but getting proper ACK going would negate their need because two way traffic would have happened recently. Better to fix underlying cause .. |
|
@gavintweedie happy to share my live code if you want to try it. Easiest way to try it is to add my fork as a HACS custom repository. It bundles the library, so it's a single install with no separate library/pip step: HACS → ⋮ → Custom repositories → add https://github.com/aramshaw/homeassistant-airtouch2plus (category: Integration) Fair warning: that fork is my working copy, so alongside the dropout fix it carries some WIP: extra docs, a few side-quests, and two features I haven't upstreamed yet (favourites read/write + a console-temperature sensor thanks to @mattjamesaus). It's not meant to be a permanent alternative to @nathanvdh, the plan is to PR everything back here (dropout fix is this PR; features to follow when he has had a chance to review), and once it's merged his repo is the home and we both go back to running his. Happy for you to run mine in the meantime just to confirm it lands the fix for you. On the ACKs: you're right that they give recent two-way traffic, and that's most of the battle. The wrinkle I hit: the controller goes quiet when nothing's changing (overnight especially), so there's nothing to ACK for stretches — I measured ~16 minutes of silence and then it drops the socket. ACKs only fire when the controller talks first, so they don't cover those idle gaps. That's what the poll is for: a lightweight status request every 4 min so the line never sits silent long enough to get dropped. Same idea as the TCP keepalives helping "a bit" — just at the app layer, so the controller's own idle timer actually resets. 3 days here with zero dropouts (was several a day before). Fair warning 2: I'm right at the limit of my understanding of Github/HACS (and quite possibly beyond it). This is in my system along side the original code and it's running. Give it a whirl and let me know how it behaves on your system. Keen to know it's not just my controller. |
Oh too easy! I have switched to your repo fork just now and disabled my automation based keepalive. I'll report back in a couple of days unless I hit a snag in the meantime. If nothing else this will give you a 2nd verification of some of your fixes and no obvious regressions. My AC system is a MHI/Mitsuibishi Heavy unit for your ref. |
|
@aramshaw Initial results are looking very good on your updated fork. I'll update in a day or two and try and resist any restarts to get a clean log comparison. Today will be the first full calendar day on your code. Yesterday (22nd) was only part day. |
|
@gavintweedie Looking good. 'unknown message' has gone to zero, so the ACK side seems to be working for you. Not perfect though. My setup has been rock solid since loading this code: no connections lost at all but I see your numbers are around the same. Let me know what a full day looks like and, if you can, if the connections lost are now a blip or if they are still minutes or hours. |
|
@aramshaw 24hrs later. No errors or disconnects, for 24hrs. |
|
@aramshaw I've been working with AI (deepseek) on the remaining disconnects and it agrees commit aa96929 (in this PR18 - re ack of 0xC0) is the best way to fix this. It's offered me some hacky ideas that could fix it in the integration but the netclient fix seems right. Honestly its a huge improvement. No unknown message types logged at all since. sed 's/\x1b[[0-9;]m//g' | |
|
Great to hear that's it's been such a big improvement and (Claude is happy) to hear deepseek independently landed on the 0xC0 fix being the right level. On the residual 2–5/day dropouts: I'm getting zero on the same code, so it smells environmental for your setup. Can you grab how long each dropout lasts? [Claude says: docker logs homeassistant 2>&1 | grep -E 'Connection lost|Reconnected' and look at the gap between each "lost" and the next "Reconnected". ] If just a few seconds it's likely a harmless blip (the client just reconnects) but minutes/hours is the kind of connection loss that the 0xC0 fix resolved for me and worth looking into more closely. Next step you could enable debug logs but there's a lot of content in them. |
|
@aramshaw I def have a 12h timout (exact 12hrs!) occuring a couple of times a day. That might be local network wifi/arp timeout or something. I'll poke around that, it doesn't appear to disconnect long and I'm not too fussed by it as this is a massive improvement where it would previously disconnect and stay disconnected until I triggered something. |
|
@gavintweedie - yeah, exactly 12hrs feels infrastructure/networky. The controller disconnects for me were variable time to drop-out, variable time to reconnect. |
3 participants
@nathanvdh - it's taken a while but here is the nicer version of PR16 raised a while ago with focused commits. It seems pretty solid from my testing.
This fixes the recurring "connection lost / unavailable" dropout. It builds
directly on your
ack-testingbranch and adds the one piece that was missing.There are two separate failure modes (they were easy to conflate)
aren't acknowledged. Your
ack-testingbranch targets this (with onecorrection, below).
socket after ~16 minutes of silence. It stops broadcasting when nothing
changes (e.g. overnight); the client then has nothing to ACK, the line goes
quiet, and ~16 min later the controller drops it. This is the "still dropped
overnight" that @gavintweedie saw when he tested
ack-testing. A lightweightstatus request every 4 minutes keeps the line warm and prevents it.
(For what it's worth: this dropout also affects the original/unmodified client —
it's the controller's behaviour, not a regression. The long-used prior code drops
the same way; it just had no "unavailable" state so it went unnoticed.)
Answering your questions from #16
"Which messages need ACKing?" — Apologies for the earlier inconsistent
lists. Verified on a live controller (firmware Console 1.2.4 / Main 2.2.0.2),
the broadcasts that require an ACK are 0x2B (SYSTEM_STATUS) and
0x45 (SYSTEM_ID). I have not observed 0x10/0x40 on my system — your
need_ackincludes them, which is harmless if they never arrive, but I can'tconfirm they need it. 0x27 (power) does not need an ACK.
"Isn't 0xC0 in the address byte redundant?" — I assumed so too, but it
turns out to be required. I tested it directly: ACKing with
0x80(thevalue
ack-testing'sAck.pyuses) makes the controller flood the sessionwith a continuous broadcast storm — ~1.7 messages/sec, ~100× normal — and it
never settles, because the ACK isn't accepted. With
0xC0it stays calm. Sothe controller genuinely validates that byte. This is almost certainly why
ack-testingdidn't behave, and it's the single change I made to yourAck.py."Why the cooldown on repeat ACKs?" — That was a defence against exactly the
storm above during testing (wrong/missing ACK → flood → don't flood ACKs back).
With the correct
0xC0ACK there's no storm, so it's unnecessary — I'veleft it out, matching your uniform approach.
"How do you know ACKs are required — did you try just sending the status
request repeatedly?" — That instinct was exactly right, and it's the second
fix here: the periodic status request (the keep-alive) is what prevents the
idle-timeout drop. And the
0x80storm above is direct proof that thecontroller actively demands the ACK.
The change (3 commits)
airtouch2/protocol: Ack messages that require it— yourack-testingcommit, kept as-is.0x80→0xC0, with the reasoning above (and yourtest_ack.pyupdated to expect it).Evidence
ack-testing(ACKs only,0x80): @gavintweedie still saw dropouts every few hours.0xC0+ keep-alive): 3+ days with zero dropouts on mysystem, across the exact overnight window that previously failed every time.
0x80ACK test: 1,516 broadcasts in 15 min (spam) vs ~1/min with0xC0.Scope
Deliberately minimal — just the dropout fix, as you suggested in #16. Favourites
(read/write) and the console-temperature sensor are working on my fork and will
follow as separate PRs once this lands.
Testing
python -m unittest discover tests/protocol— all pass, including yourtest_ack.py(updated to0xC0) and a new keep-alive test.