[ts-gen] Speculation about lost order status messages
kfmfe04 at gmail.com
Wed Oct 7 15:06:12 EDT 2009
Thank you for you comment/observation.
I suspected that I might be doing something wrong in the downstream
client - as you mentioned, no one else seems to be observing these
issues; otoh, I don't know if many users are hitting the system as
frequently as I am. Lost order status still happens to me several
times a day.
As I have been stuck too long at this last piece of getting the whole
system working, and I am not confident that I have the expertise to
manipulate low-level Ruby or C correctly, especially in the presence
of threading code, I will take a short hiatus from that part of the
Shim and see if I can get this last piece (Risk) working through the
As an aside, the Data Shim process has been running continuously and
very reliably for me.
On Oct 8, 2009, at 3:35 AM, Bill Pippin wrote:
> About your speculation as to the cause of the lost order status
> that Nils was dealing with:
>> Could this buffering/LiveLock problem potentially cause OrderStatus
>> messages from the upstream not to make it through the Shim and down
>> the Ruby client?
> No; the problem is observed as a hang, not intermittent lost messages.
> And no one has yet noted a missing order status *message* to the
> log --- Nils' problem was with silent, lost posts, which issue seems
> have been pinned down to a problem, since fixed, with duplicate keys
> interacting with the mysql "ignore" attribute in insert statements;
> "Debugging intermittent lost posts:
> Now, back to your speculations about order status messages reaching
> client. All upstream IO uses low-level IO via reads and writes to a
> socket. As I noted in your quote from my original post below, the key
> issue is lost IO events; a busybody stdlib IO layer that insists on
> buffering multiple IO events is *discarding* the initial subsequence
> those events from the viewpoint of upper layers in the IO stack, and
> non-trivial concurrent systems will not tolerate such interference.
>>> The livelock would occur where commands to the shim were
>>> in a stdlib buffer, the shim did not know of them and was loop
>>> using select for commands, and the client was loop waiting for
>>> that would not occur until the commands were processed.
> Buffering is legitimate for a layered API such as TCP/IP, where pieces
> for one layer, on one side, are buffered, or aggregated, by the *same*
> layer on the other side of the stack. E.g., network splitting and
> coalescence of packets occurs all over the Internet.
> Problems arise when some busybody library component throws away
> event notifications for a client layer higher in the stack.
> Since C stdlib IO and ruby do not do TCP/IP style encapsulation to
> distinguish the multiple IO layers --- C file descriptor, C file
> pointer, and ruby IO object --- it is up to the application developer
> to expose low-level IO state, and the related events, by eliminating
> buffering entirely for input, and limiting it to line-buffering for
> ts-general mailing list
> ts-general at trading-shim.org
More information about the ts-general