Is there a way to simultaneously write to different streams in a custom processor in NiFi? For instance I have third party libraries that do significant processing using APIs that work something like this:
public void process(InputStream in, OutputStream foo, OutputStream baa, List<String> args)
{
...
foo.write(things);
baa.write(stuff);
...
}
But the only examples I can find all just use one output stream:
FlowFile transform = session.write(original, new OutputStreamCallback() {
@Override
public void process(OutputStream out) throws IOException {
out.write("stuff");
}
});
Processing is done in batches, (due to its large scale), so its not practical to perform all the processing then write out the separate flows.
The only way I can come up with is process the input multiple times :(
To clarify, I want to write to multiple FlowFiles, using the session.write(flowfile, callback)
method, so the different streams can be sent/managed separately