[ts-gen] Keeping TWS/shim up for an entire week?

Ken Feng kfmfe04 at gmail.com
Tue Aug 18 03:58:45 EDT 2009


Hi Richard,

Thanks for pointing this out.  I thought Bill's shim-090807 release
fixed this, but it looks like only a partial fix for this problem.  If
you apply the following patch to the two files and rebuild, you should
get 5 decimals:

Regards,
Ken

lib/text.c
141c141
< Sig(double) { append("%.5f", t);              return self; }
---
> Sig(double) { append("%.4f", t);              return self; }
------------------------------------------------------------------------
src/tags.c
128,131c128,131i
<       Open(*new(perm) tag:: Dec(e, t.dec   , "open" ,      "%11.5f")),
<       High(*new(perm) tag:: Dec(e, t.dec   , "high" ,      "%11.5f")),
<        Low(*new(perm) tag:: Dec(e, t.dec   , "low"  ,      "%11.5f")),
<       Clos(*new(perm) tag:: Dec(e, t.dec   , "close",      "%11.5f")),
---
>       Open(*new(perm) tag:: Dec(e, t.dec   , "open" ,      "%8.2f")),
>       High(*new(perm) tag:: Dec(e, t.dec   , "high" ,      "%8.2f")),
>        Low(*new(perm) tag:: Dec(e, t.dec   , "low"  ,      "%8.2f")),
>       Clos(*new(perm) tag:: Dec(e, t.dec   , "close",      "%8.2f")),
------------------------------------------------------------------------


On 8/18/09, Richard Pruss <boadie at gmail.com> wrote:
> Ken,
> How are you getting the addition decimals on forex bars, your output
> seems to imply 3 on YENAUD?  I only seem to get two decimal places in
> the ShimText, regardless of currency.
>
> 49076|59268|1577398571|3|50| 3|    6|1250576860|   78.49|   78.49|
> 78.47|   78.47|    -1|   -1.00|    -1|CASH::AUD:JPY:
> 49076|59270|1579549294|3|50| 3|    4|1250576865|    0.83|    0.83|
> 0.83|    0.83|    -1|   -1.00|    -1|CASH::AUD:USD:
> 49076|59270|1579550410|3|50| 3|    5|1250576865|    0.83|    0.83|
> 0.83|    0.83|    -1|   -1.00|    -1|CASH::AUD:USD:
>
> Clearly on something like AUDUSD one needs 4 decimal places, to do
> something useful with the log.
>
> -Ric
>
> On Tue, Aug 18, 2009 at 3:58 PM, Ken Feng<kfmfe04 at gmail.com> wrote:
>> Hi Bill & Russ,
>>
>> Regarding my daily 3:55UTC problem,
>>
>>> 1.  For example, I know for a fact that around 3:55UTC every single
>>> day, I get an odd behavior in the system where the TWS stops
>>> fulfilling the bar requests, even though TWS itself still receives
>>> ticks and continues to display them in the GUI.  When I kill the shim
>>> and restart it manually, the response has been sporadic - sometimes,
>>> it works on the first try.  Sometimes on the second or third try.  My
>>> guess is, the TWS has fallen into a state that I have not yet
>>> anticipated, for whatever reasons (maybe daily maintenance by IB?),
>>> and I need to to either detect the state at any time or patch around
>>> the 3:55UTC gap.
>>
>> following Bill's suggestion that I cancel the bar requests first, I
>> was able to continue receiving ticks without manual intervention.  The
>> gap, however is kind of large:
>>
>> AUDJPY|2009-08-18 03:54:55|77.960|77.960|77.960|77.960
>> AUDJPY|2009-08-18 03:55:00|77.960|77.960|77.960|77.960
>> AUDJPY|2009-08-18 04:05:25|78.040|78.050|78.040|78.040
>> AUDJPY|2009-08-18 04:05:30|78.040|78.040|78.040|78.040
>>
>> I dug into ShimText and found the following in summarized form (table
>> lines up in a monospaced font:
>>
>> ----------------------------------------------------------------------
>> secs  utime      utc      comment
>> ----------------------------------------------------------------------
>> 46505 1250567700 03:55:00 Last Good Tick at 03:55:00
>> 46509                     MDS data farm connection is broken:ushmds2a
>> 46512 ----- TRIGGER ----- MDS data farm connection is OK:ushmds2a
>> 46512                     Ruby: begin sending 15 cancel bars to shim
>> 46527                     Ruby: begin sending 15 select bars to shim
>> 46861                     Market data farm connection is broken:cashfarm
>> 46939                     Market data farm connection is OK:cashfarm
>> 47111 ----- TRIGGER ----- MDS data farm connection is OK:ushmds2a
>> 47111                     Ruby: begin sending 15 cancel bars to shim
>> 47113 1250568305 04:05:05 EURCHF, EURJPY, USDCAD, EURUSD, USDJPY, EURGBP,
>>                          EURCAD, GBPCHF, NZDUSD, USDCHF, GBPUSD, GBPJPY
>> 47126                     Ruby: begin sending 15 select bars to shim
>> 47130 1250568325 04:05:25 AUDJPY, AUDUSD, and others come in
>> ----------------------------------------------------------------------
>>
>> I am TRIGGERing 15 cancel bars and 15 select bars to be sent to the
>> shim on "MDS data farm connection is OK:ushmds2a", and even though
>> this works, the behavior appears to be sub-optimal.
>>
>> 1. Observe 47113 - the fact that I am already receiving ticks here
>> implies that 46512 and 46527 are sufficient, while 47111 and 47126
>> appear to be redundant.
>> 2. If I use no TRIGGERs at all, I will fail to receive any ticks at
>> all after 46505.
>>
>> So given 1. and 2., we now have some state-based ugliness.  It almost
>> seems that instead of TRIGGERing on "MDS data farm connection is
>> OK:ushmds2a", I must TRIGGER if and only if "Market data farm
>> connection is broken:cashfarm" and "Market data farm connection is
>> OK:cashfarm" do not immediately proceed it.
>>
>> I think this kind of state-based code should work to prevent 47111 and
>> 47126 redundancies, but it is extremely brittle and prone to breakage,
>> and difficult to debug when the server changes.  I can go for
>> "tight-coding" where the state-based code is expected to work only for
>> this particular case, or I can go for "general-coding" where it will
>> apply to similar events throughout the day.  Without any
>> specifications from IB, I can only guess which type of coding may be
>> more "robust".  At this point, I am towards going with the
>> "tight-coding", perhaps even limiting the TRIGGER to a certain range
>> of UTC time.
>>
>> If anyone has any better, simple solutions, please do let me know!
>>
>> BTW, Russ' suggestion of modifying .shimrc values like ShimText before
>> shim startup also works very well.  I now make my Ruby script modify
>> the .shimrc to output to yyyymmdd.log before starting up the shim.
>>
>> Thank you.
>>
>> - Ken
>> _______________________________________________
>> ts-general mailing list
>> ts-general at trading-shim.org
>> http://www.trading-shim.org/mailman/listinfo/ts-general
>>
>
> _______________________________________________
> ts-general mailing list
> ts-general at trading-shim.org
> http://www.trading-shim.org/mailman/listinfo/ts-general
>


More information about the ts-general mailing list