I got the push but the chat was empty
Recurring complaint from myself: the assistant pushes a notification that it's done, I open the app, the conversation is there, the assistant bubble is empty. Pull-to-refresh fills it in. Background sync never does.
What was happening
The assistant streams its response over SSE while a long-running
tool job churns in the background. When the iOS app gets
backgrounded mid-stream, it disconnects from SSE and falls back to
polling /conversations/{id}/poll. That endpoint returns:
changed = (conv.updated_at > since)
If changed is false, the client doesn't re-fetch /messages. The
push notification arrives at job-end via APNs, the user opens the
app, but updated_at looks the same as it did when the user closed
the app — so the client never refetches and the bubble stays empty.
What I found
HistoryStore.append_to_message was updating the message row's
content field on every chunk, but it never touched the
conversation's META row. updated_at only got bumped by
add_message (called once at the start of the turn, with empty
content) and by the end-of-job touch().
So during the actual streaming window — when the bubble is filling
up — the conversation's updated_at was frozen. The poll endpoint
correctly reported "nothing changed" because the field it was
checking truly hadn't moved.
The fix
One DynamoDB UpdateItem per chunk. Cheap.
async def append_to_message(self, pk: str, msg_id: str, chunk: str):
# existing UpdateItem on the message row
await self._update_message_content(pk, msg_id, chunk)
# new: bump the META row so pollers learn there's new content
await self.touch(pk.removeprefix("CONV#"))
touch() was already there for the end-of-job bump; it sets
updated_at and gsi1sk together so the sidebar's recency sort
also moves the conversation to the top during streaming.
What I'd do differently
The SSE handler and the poll handler are two different code paths
agreeing on a shared invariant — "any visible change bumps
updated_at." Nothing in the codebase enforced that invariant. I'd
add a test that streams a chunk and then asserts the conversation
META row's updated_at moved, so the next person who writes a new
append path doesn't drop the touch and reintroduce the same ghost
bubble.