Re: Gemini API finishReason RECITATION - Google Cloud Community
Hi all, I use Gemini through its API on https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent. I sometimes get 0 candidates, with a finishReason = RECITATION. What should be the parameters in the POST query if I don't want it to stop for that reason but still answer? I see that there is a Github Issue about that, here: https://github.com/google/generative-ai-docs/issues/257. Thank you!
Streaming Responses for Better Detection
If I prompt Gemini via the API with this, I hit the RECITATION issue. But if I paste it on the Gemini website, I get my answer: https://gemini.google.com/app/11a4c6b2bdcf6b7f. I think this is unfair...
Optimizing Response Detection
If I prompt Gemini via the API with this, I hit the RECITATION issue. But if I paste it on the Gemini website, I get my answer: https://gemini.google.com/app/11a4c6b2bdcf6b7f. I think this is unfair...
Stream the response if you are not already; it will detect way less recitation than if you get it all at once.
With simple queries, like adjusting the number of JSON objects sent, modifying multiprocessing settings, or running the code repeatedly, it fails with vertexai.generative_models._generative_models.ResponseValidationError: The model response did not complete successfully. Finish reason: 4. Gemini Ultra is totally unusable.
Troubleshooting RECITATION Errors
I'm also encountering the block reason RECITATION error when using gemini-1.0-pro-001; setting: BLOCK_NONE with temperature=0.2 & t-top=0.6. I've narrowed down the issue to a specific day of data. Interestingly, I was able to get the needed response by:
- Adjusting the number of JSON objects sent
- Modifying multiprocessing settings
- Running the code repeatedly for that day
However, this seems like a workaround, and the root cause of the error remains unclear.
Enhancing Chat Interactions
Same issue here by simply asking it to explain a simple concept. Do I need to stream responses for simple chat interactions?
Same case here. I asked a question "What is information security?". Then, recitation error.
I've ceased attempts to utilize Google language models through Vertex AI as they proved to be non-functional. No feedback from Google.
Issues with RAG Implementation
Did you manage to get anywhere with this? I'm trying to build RAG on top of this, but it keeps showing this error randomly. It's so frustrating.
Nope, my company just stopped trying to use Vertex AI. It was a terrible experience.
Same case here. Still facing the same issue with gemini-1.5-flash!
Unreliability of Responses
Yes, very unstable. I used a lot of JSON object outputs. Response out contains extractions, a lot of time and made an invalid JSON format. Then, my program crashed. So, annoying. However, it is smarter than GPT-4o mini a little bit. So, I still keep using gemini-1.5 flash.
