So finally, we were able to find this data in the buffered messages. The issue we are now running into is that the way we read the files breaks the buffered MessageBroadcaster. When we read a file, we do the following:
1. Set the incompleteStream flag to true. This allows the decoder to read files that are incomplete -- or cutoff at the end.
2. Skip reading of the header. The header contains a checksum in it. If the checksum fails, the file won't process. Skipping the header ignores that checksum check, and allows us to process what data is present.
If we don't do #1 and #2, we can read the buffered messages, but then we can't parse as many files (we find a lot of whacky files that these methods allow us to parse correctly). I guess we have to decide which route we want to go.
Does anyone know how common/uncommon these buffered messages are? I guess that would help us decide if we no longer do #1 and #2 so we can get the buffered data, or if we leave things as they are (assuming the buffered data in files is fairly rare).
