bullitt/trackbox
1
0
Fork 0
Code Issues 21 Pull requests Projects Releases 46 Packages Wiki Activity Actions

[WSJF-4] Add few-shot calibration examples to AI extractor prompt #13

New issue
Open
opened 2026-06-27 10:45:50 +02:00 by bullitt · 0 comments
bullitt commented 2026-06-27 10:45:50 +02:00
Owner
Copy link

Context

Article: 'Few-shot calibration ensures evaluator judgment aligns with preferences.'

Problem

Our AI prompt has no examples. The model guesses the desired output format from instructions alone, leading to inconsistent quality.

Proposal

  • Add 3 few-shot examples to SYSTEM_PROMPT:
    1. DHL 'ist unterwegs' email → expected extraction + field_map
    2. Amazon 'Geliefert' email → expected extraction + field_map
    3. Hermes 'auf dem Weg' email → expected extraction + field_map
  • Ensures consistent field naming, title style, and field_map strategy choices

WSJF: 4

## Context Article: 'Few-shot calibration ensures evaluator judgment aligns with preferences.' ## Problem Our AI prompt has no examples. The model guesses the desired output format from instructions alone, leading to inconsistent quality. ## Proposal - Add 3 few-shot examples to SYSTEM_PROMPT: 1. DHL 'ist unterwegs' email → expected extraction + field_map 2. Amazon 'Geliefert' email → expected extraction + field_map 3. Hermes 'auf dem Weg' email → expected extraction + field_map - Ensures consistent field naming, title style, and field_map strategy choices ## WSJF: 4
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
architecture-rework
deploy-20260710-47cffe0
v0.17.3
deploy-20260710-05ae584
v0.17.2
deploy-20260710-e0b0541
v0.17.1
deploy-20260710-9965291
v0.17.0
deploy-20260706-1d76568
v0.16.0
deploy-20260706-feb7fc6
v0.15.0
deploy-20260706-a51b50a
v0.14.0
deploy-20260706-b58f02a
deploy-20260706-44736ed
deploy-20260706-011f7b8
v0.13.15
deploy-20260705-0cc85b9
deploy-20260705-e584012
deploy-20260705-bd8a500
deploy-20260705-bfd3e98
v0.13.14
deploy-20260705-e87c80f
deploy-20260705-cda820e
deploy-20260705-be80bd0
v0.13.13
deploy-20260705-578e35c
deploy-20260705-67a4a1f
deploy-20260705-b462d1d
deploy-20260705-1c4e0d4
v0.13.12
deploy-20260705-a79acac
v0.13.11
deploy-20260705-5ad9a2e
v0.13.10
deploy-20260705-2801cbe
v0.13.9
deploy-20260705-d4354f9
v0.13.8
deploy-20260705-d54653b
v0.13.7
deploy-20260705-db38468
deploy-20260705-dabdbc2
deploy-20260705-e3c857f
v0.13.6
deploy-20260704-4afffd7
v0.13.5
deploy-20260704-71064fc
v0.13.4
deploy-20260704-a46888f
v0.13.3
deploy-20260704-881af7d
v0.13.2
deploy-20260704-0f744e4
v0.13.1
deploy-20260704-f637348
v0.13.0
deploy-20260704-4fa215f
deploy-20260704-2b71d11
v0.12.0
deploy-20260704-a04faec
v0.11.0
deploy-20260704-86d3e07
v0.10.5
deploy-20260703-beada7b
v0.10.4
deploy-20260703-afcf8e7
v0.10.3
deploy-20260703-9a7d76d
v0.10.2
deploy-20260703-d3d396f
v0.10.1
deploy-20260703-121e400
v0.10.0
deploy-20260703-810a446
v0.9.0
deploy-20260701-bdbb181
v0.8.1
deploy-20260701-d465784
v0.8.0
deploy-20260630-b30dac3
v0.7.0
deploy-20260630-9537c98
deploy-20260630-0b0bb27
v0.6.0
deploy-20260630-4dc7da9
v0.5.1
deploy-20260630-1b8e601
v0.5.0
deploy-20260630-cd259f8
v0.4.0
deploy-20260630-d2cc0db
v0.3.1
deploy-20260630-a8d6c14
v0.3.0
deploy-20260630-1e4b257
v0.2.4
deploy-20260630-f940733
v0.2.3
deploy-20260630-bf4abb2
deploy-20260629-d2590fa
deploy-20260629-7a38387
deploy-20260629-457b981
deploy-20260628-eaf1e5a
deploy-20260628-9cdb980
deploy-20260628-3e77cd7
deploy-20260628-a923aee
deploy-20260628-1941f54
deploy-20260628-38c05ac
deploy-20260628-1700eb4
deploy-20260628-f7e8243
deploy-20260628-e12e7e8
deploy-20260628-9e66346
deploy-20260628-65b72d6
deploy-20260628-77ed37e
deploy-20260628-ce89163
deploy-20260628-4e14a1d
deploy-20260628-0273f4e
deploy-20260628-1b402e9
deploy-20260628-589cac3
deploy-20260628-4a7dbfe
deploy-20260628-8c315ab
deploy-20260628-23a69dc
deploy-20260628-6e00b0b
deploy-20260628-a1bc1af
deploy-20260628-356fb91
deploy-20260628-03f41a7
deploy-20260628-0f9d178
deploy-20260628-d006577
deploy-20260628-523d176
deploy-20260628-b7197c1
deploy-20260628-a5533bf
deploy-20260628-1a18864
deploy-20260628-81f3728
deploy-20260628-fc5b42b
deploy-20260628-b0562bd
deploy-20260628-07c4528
deploy-20260628-f3672e5
deploy-20260628-6184d7a
deploy-20260628-a47ffca
deploy-20260628-622dc6a
deploy-20260628-222f21d
deploy-20260628-efdf448
deploy-20260628-9048059
deploy-20260628-a589599
deploy-20260628-bdd22df
deploy-20260628-857f889
deploy-20260628-0253b56
deploy-20260628-611b704
deploy-20260628-c40aa51
deploy-20260628-e5ce47f
deploy-20260628-9e0041e
deploy-20260628-2f77539
deploy-20260628-b4b8126
deploy-20260628-52d7506
deploy-20260628-b52b8bd
deploy-20260628-56bfeb3
deploy-20260628-e7d750a
deploy-20260628-0e9cbdb
deploy-20260628-80ff048
deploy-20260628-3784158
deploy-20260628-f97ae48
deploy-20260628-6cf7d20
deploy-20260628-4b4899c
deploy-20260628-0f01c0a
deploy-20260628-0b7b0e5
deploy-20260628-a2b65de
deploy-20260628-1884045
deploy-20260628-13a31e1
deploy-20260628-842c11d
deploy-20260628-632cebd
deploy-20260627-ca2bfb0
deploy-20260627-b22b981
deploy-20260627-c39b6db
deploy-20260627-17a6689
deploy-20260627-90844c7
deploy-20260627-1624f30
deploy-20260627-4833785
deploy-20260627-32a7a3b
deploy-20260627-badc4ae
deploy-20260627-9dd1448
deploy-20260627-aa0b43e
deploy-20260627-16c7ab5
deploy-20260627-99f813a
v0.1.1
deploy-20260627-938b1ff
deploy-20260627-2da47f7
v0.2.0
deploy-20260627-7b96660
v0.1.0
deploy-20260627-00968e7
deploy-20260627-b11f17b
deploy-20260627-a4987a7
deploy-20260627-c6b0e0f
deploy-20260627-76bfc82
deploy-20260627-bd6b0e7
deploy-20260627-507da61
deploy-20260627-c4c2c44
deploy-20260627-ee354d5
deploy-20260627-7eadb39
deploy-20260627-632dde5
deploy-20260627-cd2d45b
deploy-20260627-1253b03
deploy-20260627-7df6dee
deploy-20260627-a4517fe
deploy-20260627-a24d734
deploy-20260627-abe083d
deploy-20260627-a7abd5c
deploy-20260627-7a4b0e2
deploy-20260627-766e0a2
deploy-20260627-e11b7e9
deploy-20260627-3f8a85a
deploy-20260627-4b87dc6
deploy-20260627-7bd6416
deploy-20260627-b2780e8
deploy-20260627-ecd7f38
deploy-20260627-baa1126
deploy-20260627-8a05b5e
deploy-20260627-6ce1fa8
deploy-20260627-c6fe30a
deploy-20260627-e64e7d3
deploy-20260627-3b285dc
deploy-20260627-0d5869a
deploy-20260627-12d83c4
deploy-20260627-8e0dc62
deploy-20260627-a1d0c96
deploy-20260627-1693222
deploy-20260627-8ae1693
deploy-20260627-5e63ebd
deploy-20260627-e51193b
deploy-20260627-bcc6f22
deploy-20260627-6ed2b82
deploy-20260627-9d7babc
deploy-20260627-5e6162e
deploy-20260627-dbe3edf
deploy-20260627-35f5165
deploy-20260627-115536d
deploy-20260627-2273884
Labels
Clear labels   
arch

Architecture review findings (docs/architecture-review.md)

  
harness

Harness design patterns from Anthropic engineering article

  
security

Security vulnerabilities and hardening

  
ux

User experience and interface improvements

No labels
arch
harness
security
ux
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
bullitt/trackbox#13
No description provided.