Hindsight/Benchmarks

Community Docs GitHub

LoComo Benchmark

Long conversation memory evaluation

Overall

Accuracy

89.61%

Correct

1380 / 1540

Conversations

10

By Category

Multi-hop

86.2%

243 / 282

Single-hop

83.8%

269 / 321

Temporal

70.8%

68 / 96

Open-domain

95.1%

800 / 841

Conversations (10)

conv-26

94.1%143/152 correct

conv-30

92.6%75/81 correct

conv-48

90.6%173/191 correct

conv-47

91.3%137/150 correct

conv-44

87.8%108/123 correct

conv-43

85.4%152/178 correct

conv-50

89.9%142/158 correct

conv-42

87.9%175/199 correct

conv-49

86.5%135/156 correct

conv-41

92.1%140/152 correct