feat: more graceful handling of API call didn't return a message message.

[sarahwooders]

When the model fails to generate a message for send_message, we should re-try or give a better errror:

Letta.letta.server.server - ERROR - Error in server._step: API call didn't return a message: id='message-9fdf48af-b625-46f0-8b41-68bc3131d7f5' choices=[] created=datetime.datetime(2024, 9, 26, 6, 40, 38, 163866, tzinfo=datetime.timezone.utc) model='gpt-4o-2024-05-13' system_fingerprint='fp_e375328146' object='chat.completion' usage=UsageStatistics(completion_tokens=48, prompt_tokens=3699, total_tokens=3747)

[kurisu]

@sarahwooders Thank you for capturing this. I'm happy to provide context, and have some suggestions:

1. Since this error seems to be caused by LLMs failing to format function calls correctly, you could include a message to that effect in the error message so folks can see that possibility in the log.
2. Since some LLMs are known by the community to be compatible (or not) with Letta, you could update the documentation to say so, and list known-good LLMs so folks don't just try newer models like gpt-4o that have generally good reputations but don't work well with Letta.
3. @swooders. There's a gotcha: just because the config file says gpt-4 doesn't mean Agents will use it. For example, I had already created an agent using gpt-4o, then switched to gpt-4 in the config. I then tried the previously created Agent, thinking it would use the setting from the config file. It did not. The Agent overrode the config and continued to use gpt-4o. I only got it working after I updated both the config AND created a new agent which also (redundantly) specifies gpt-4.

Suggestion: I wish it was more clear in the UX that Agents override the model from the config. In fact, nowhere on the Agents dashboard or Agent Chat UIs does it mention what model is being used. Expanding an Agent in the dashboard only shows Agent Persona and Human Persona. I had to notice in the logs that gpt-4o was still being used in order to figure out the config was being overridden by the agent. Also, it would be helpful if Agents could be edited and deleted in the ADE. I've had to just create new ones repeatedly to get to a stable working state, and now there's cruft I can't get rid of. I'm not even sure where Agents are stored, and I don't see any config files for them in ~/.letta/agents/

[distributev]

regarding this message sometimes I got the impression that this error / warning came even though a return message was actually returned (because I see the returned message in the stack trace)
most likely something can be improved here at least in give some better feedback to the user, if not more.