[ts-gen] Keeping TWS/shim up for an entire week?

Richard Pruss boadie at gmail.com
Tue Aug 18 19:21:13 EDT 2009


Thanks Ken,  I am now able to see all 5 decimals.

- Ric

On 18/08/2009, at 5:58 PM, Ken Feng wrote:

> Hi Richard,
>
> Thanks for pointing this out.  I thought Bill's shim-090807 release
> fixed this, but it looks like only a partial fix for this problem.  If
> you apply the following patch to the two files and rebuild, you should
> get 5 decimals:
>
> Regards,
> Ken
>
> lib/text.c
> 141c141
> < Sig(double) { append("%.5f", t);              return self; }
> ---
>> Sig(double) { append("%.4f", t);              return self; }
> ------------------------------------------------------------------------
> src/tags.c
> 128,131c128,131i
> <       Open(*new(perm) tag:: Dec(e, t.dec   , "open" ,       
> "%11.5f")),
> <       High(*new(perm) tag:: Dec(e, t.dec   , "high" ,       
> "%11.5f")),
> <        Low(*new(perm) tag:: Dec(e, t.dec   , "low"  ,       
> "%11.5f")),
> <       Clos(*new(perm) tag:: Dec(e, t.dec   , "close",       
> "%11.5f")),
> ---
>>      Open(*new(perm) tag:: Dec(e, t.dec   , "open" ,      "%8.2f")),
>>      High(*new(perm) tag:: Dec(e, t.dec   , "high" ,      "%8.2f")),
>>       Low(*new(perm) tag:: Dec(e, t.dec   , "low"  ,      "%8.2f")),
>>      Clos(*new(perm) tag:: Dec(e, t.dec   , "close",      "%8.2f")),
> ------------------------------------------------------------------------
>
>
> On 8/18/09, Richard Pruss <boadie at gmail.com> wrote:
>> Ken,
>> How are you getting the addition decimals on forex bars, your output
>> seems to imply 3 on YENAUD?  I only seem to get two decimal places in
>> the ShimText, regardless of currency.
>>
>> 49076|59268|1577398571|3|50| 3|    6|1250576860|   78.49|   78.49|
>> 78.47|   78.47|    -1|   -1.00|    -1|CASH::AUD:JPY:
>> 49076|59270|1579549294|3|50| 3|    4|1250576865|    0.83|    0.83|
>> 0.83|    0.83|    -1|   -1.00|    -1|CASH::AUD:USD:
>> 49076|59270|1579550410|3|50| 3|    5|1250576865|    0.83|    0.83|
>> 0.83|    0.83|    -1|   -1.00|    -1|CASH::AUD:USD:
>>
>> Clearly on something like AUDUSD one needs 4 decimal places, to do
>> something useful with the log.
>>
>> -Ric
>>
>> On Tue, Aug 18, 2009 at 3:58 PM, Ken Feng<kfmfe04 at gmail.com> wrote:
>>> Hi Bill & Russ,
>>>
>>> Regarding my daily 3:55UTC problem,
>>>
>>>> 1.  For example, I know for a fact that around 3:55UTC every single
>>>> day, I get an odd behavior in the system where the TWS stops
>>>> fulfilling the bar requests, even though TWS itself still receives
>>>> ticks and continues to display them in the GUI.  When I kill the  
>>>> shim
>>>> and restart it manually, the response has been sporadic -  
>>>> sometimes,
>>>> it works on the first try.  Sometimes on the second or third  
>>>> try.  My
>>>> guess is, the TWS has fallen into a state that I have not yet
>>>> anticipated, for whatever reasons (maybe daily maintenance by IB?),
>>>> and I need to to either detect the state at any time or patch  
>>>> around
>>>> the 3:55UTC gap.
>>>
>>> following Bill's suggestion that I cancel the bar requests first, I
>>> was able to continue receiving ticks without manual intervention.   
>>> The
>>> gap, however is kind of large:
>>>
>>> AUDJPY|2009-08-18 03:54:55|77.960|77.960|77.960|77.960
>>> AUDJPY|2009-08-18 03:55:00|77.960|77.960|77.960|77.960
>>> AUDJPY|2009-08-18 04:05:25|78.040|78.050|78.040|78.040
>>> AUDJPY|2009-08-18 04:05:30|78.040|78.040|78.040|78.040
>>>
>>> I dug into ShimText and found the following in summarized form  
>>> (table
>>> lines up in a monospaced font:
>>>
>>> ----------------------------------------------------------------------
>>> secs  utime      utc      comment
>>> ----------------------------------------------------------------------
>>> 46505 1250567700 03:55:00 Last Good Tick at 03:55:00
>>> 46509                     MDS data farm connection is  
>>> broken:ushmds2a
>>> 46512 ----- TRIGGER ----- MDS data farm connection is OK:ushmds2a
>>> 46512                     Ruby: begin sending 15 cancel bars to shim
>>> 46527                     Ruby: begin sending 15 select bars to shim
>>> 46861                     Market data farm connection is  
>>> broken:cashfarm
>>> 46939                     Market data farm connection is OK:cashfarm
>>> 47111 ----- TRIGGER ----- MDS data farm connection is OK:ushmds2a
>>> 47111                     Ruby: begin sending 15 cancel bars to shim
>>> 47113 1250568305 04:05:05 EURCHF, EURJPY, USDCAD, EURUSD, USDJPY,  
>>> EURGBP,
>>>                         EURCAD, GBPCHF, NZDUSD, USDCHF, GBPUSD,  
>>> GBPJPY
>>> 47126                     Ruby: begin sending 15 select bars to shim
>>> 47130 1250568325 04:05:25 AUDJPY, AUDUSD, and others come in
>>> ----------------------------------------------------------------------
>>>
>>> I am TRIGGERing 15 cancel bars and 15 select bars to be sent to the
>>> shim on "MDS data farm connection is OK:ushmds2a", and even though
>>> this works, the behavior appears to be sub-optimal.
>>>
>>> 1. Observe 47113 - the fact that I am already receiving ticks here
>>> implies that 46512 and 46527 are sufficient, while 47111 and 47126
>>> appear to be redundant.
>>> 2. If I use no TRIGGERs at all, I will fail to receive any ticks at
>>> all after 46505.
>>>
>>> So given 1. and 2., we now have some state-based ugliness.  It  
>>> almost
>>> seems that instead of TRIGGERing on "MDS data farm connection is
>>> OK:ushmds2a", I must TRIGGER if and only if "Market data farm
>>> connection is broken:cashfarm" and "Market data farm connection is
>>> OK:cashfarm" do not immediately proceed it.
>>>
>>> I think this kind of state-based code should work to prevent 47111  
>>> and
>>> 47126 redundancies, but it is extremely brittle and prone to  
>>> breakage,
>>> and difficult to debug when the server changes.  I can go for
>>> "tight-coding" where the state-based code is expected to work only  
>>> for
>>> this particular case, or I can go for "general-coding" where it will
>>> apply to similar events throughout the day.  Without any
>>> specifications from IB, I can only guess which type of coding may be
>>> more "robust".  At this point, I am towards going with the
>>> "tight-coding", perhaps even limiting the TRIGGER to a certain range
>>> of UTC time.
>>>
>>> If anyone has any better, simple solutions, please do let me know!
>>>
>>> BTW, Russ' suggestion of modifying .shimrc values like ShimText  
>>> before
>>> shim startup also works very well.  I now make my Ruby script modify
>>> the .shimrc to output to yyyymmdd.log before starting up the shim.
>>>
>>> Thank you.
>>>
>>> - Ken
>>> _______________________________________________
>>> ts-general mailing list
>>> ts-general at trading-shim.org
>>> http://www.trading-shim.org/mailman/listinfo/ts-general
>>>
>>
>> _______________________________________________
>> ts-general mailing list
>> ts-general at trading-shim.org
>> http://www.trading-shim.org/mailman/listinfo/ts-general
>>
> _______________________________________________
> ts-general mailing list
> ts-general at trading-shim.org
> http://www.trading-shim.org/mailman/listinfo/ts-general



More information about the ts-general mailing list