Use Sampler for StaticWhisperPipeline #1713
Conversation
eshiryae
changed the title
| std::vector<int64_t> chunk_init_tokens; | ||
| chunk_init_tokens.insert(chunk_init_tokens.end(), init_ids.begin(), init_ids.end()); | ||
|
|
||
| SequenceGroup::Ptr sequence_group = std::make_shared<SequenceGroup>(0, chunk_init_tokens, config, 1); |
There was a problem hiding this comment.
chunk_init_tokens looks redundant. Can't we pass init_ids directly?
| return {false, output_tokens}; | ||
| std::unordered_map<uint64_t, ov::genai::GenerationOutput> token = handle->read(); | ||
| for (const auto& gen_token : token.begin()->second.generated_ids) { | ||
| if (streamer_ptr->put(gen_token)) { |
There was a problem hiding this comment.
fyi streamer->put is deprecated. Stateful pipeline reference: https://github.com/openvinotoolkit/openvino.genai/blob/master/src/cpp/src/whisper/whisper.cpp#L70
| if (output_token == config.eos_token_id) { | ||
| break; | ||
| } | ||
| while (!sequence_group->has_finished() && !sequence_group->handle_dropped()) { |
There was a problem hiding this comment.
handle_dropped is outdated, please, use
(!sequence_group->has_finished() && !sequence_group->handle_stopped() && !sequence_group->handle_cancelled()), example: https://github.com/openvinotoolkit/openvino.genai/blob/master/src/cpp/src/whisper/whisper.cpp#L107
| return {true, output_tokens}; | ||
| } | ||
| } | ||
| OPENVINO_ASSERT(sequence_group->get_finished_sequences().size() == 1u); |
There was a problem hiding this comment.
What situation can that there will be more than one sequence here?
There was a problem hiding this comment.
Such cases not supported for StaticWhisper pipeline yet
CVS-161218