Some bots occasionally take a really long time to run. I'm not sure why.
Examplest errors
- link
- link
- Figure out why deepseek v3_2 has timeout. Does it really take > 3min to forecast? link
Most of this has been solved with longer timeouts per response. But a bot shouldn't need 5min to make a forecast I don't think. This makes me think I'm doing something wrong.
Hypotheses
- It actually takes bots this long
- The Metaculus Client has a sync timeout when waiting between requests in order to avoid rate limiting. If this triggers when one forecast finishes, then the other forecasts wait a while to finish (though the long timeouts happened even before the rate limiting change).
- Something to do with the concurrency limit 1 for research
- We are using 2 structure_outputs per forecast
- We messed up batching somewhere
Some bots occasionally take a really long time to run. I'm not sure why.
Examplest errors
Most of this has been solved with longer timeouts per response. But a bot shouldn't need 5min to make a forecast I don't think. This makes me think I'm doing something wrong.
Hypotheses