It can happen that we already cleared libvchan_fd pending state via
libvchan_wait, but data arrived later. This is especially true just
after connection, when client send unsolicited notification to server,
which can confuse it with some requested notification.