Integration Data queues, or How FedEx is killing our performance with status updates

I inherited our implementation of OTM 6.1.7 about a year ago from OCS. Our system is rather simple compared to many OTM implementations. We use OTM mainly for single leg planning and work flow of Small Parcel, LTL, and TL shipments only. (We do have extreme complexities elsewhere, but that's for another discussion.) Our OTM implementation is relatively stable and we are aggressively maturing our OTM knowledge.
Our implementation includes live updates (edi 214) from all of our carriers. Our VAN translates the EDI into OTM specific XML and sends them to OTM with out a problem. This occurs randomly through out the day and works well. However, from time to time FedEx will drop an enormous amount of 214's on us and OTM comes to a crawl. I attended the 2014 SIG conference and during one of the open forums someone brought up this exact issue and Integration Data Queues were brought up as a potential solution. (If you haven't gone to the OTM SIG, make a point to do so!! I know I'll be there next year if I can.) I had never heard of this subject and discussed it, at a high level, with a few folks with a plan to implement this potential God send of a solution as soon as I got back. Well, it has been a week now and I believe I have Data Queue configured in our DEV environment but have a hard time believing it is this simple. Thus the thread. (No, I didn't start this just to talk. :))
So, is the Integration in Direct XML Data Queue really this simple to implement? No glog.properties settings?!?? Have I missed something?




As DBA.Admin I navigated to the Data Queue Manager and configured the Integration In Direct XML in the following manner.

• Active - Check
• Thread Count - 1
• Batch Size - 50
• Polling Frequency - 1 minute (I realize this may be extreme and will modify in PROD once implemented and monitored)
• Data Queue Table - INTEGRATION_IN
• Data Queue Poller - INTEGRATION IN DIRECT XML
• Data Queue Executor - INTEGRATION IN DIRECT XML
• Finder Set - Q_INTEGRATION_IN_DIRECT

Is it REALLY this simple? If so, I have to say this MUCH more simplistic than any other process I've had to figure out, on my own, in OTM.

Lastly, can anyone expound on the Related Queue expected usage?


Further questions to ask once a brave soul wanders in and decides to speak up.

1. How can you be sure the configuration is working?
2. How to identify the optimum Thread Count, batch size, Polling frequency? (Event Diag Servlet discussion expected to get a bit nutty here.)
3. Is there a specific Data Queue Servlet? (For the record, I did search OTMFAQ and OTMWiki)
4. Are Data Queues a "set it and forget it" paradigm or something that should be reviewed on a timely basis? If timely, what frequency?
5. How does one know when the Data Queue reaches diminishing returns?
6. etc.....