Piloting a corpus-based CEU workshop

Has anyone implemented CEU activities that use corpus annotation to target ASL prosody and classifier depiction? I’m trialing a 90-minute session in November where participants use ELAN to mark boundary placement, holds, eye-gaze shifts, and mouth posture categories on four 90-second clips, and I’d value input on the learning outcomes and a practical scoring rubric for reliability.

‌⁠‍⁠​‍​‍‌⁠‌​​‍​‍​⁠‍‍​‍​‍‌⁠​⁠‌‍‍‌‌‍‌⁠‌‍⁠‍‌‍⁠​‌‍​‌‌‍⁠‍‌‍‌⁠‌⁠‌‌‌‍​‌‌‍‌⁠‌‍‌‌‌‍‍‌‌‍⁠‍‌⁠‌​‌‍‌‌‌⁠​‍‌⁠​​‌⁠​‍‌‍‌‌‌⁠‌​‌‍‌‌‌⁠​‍​‍​‍​‍⁠​​‍​‍‌‍‍⁠​‍​‍​⁠‍‍​‍​‍‌‍⁠‍‌‍‌‌‌⁠‌⁠‌‌⁠⁠‌⁠‌​‌‍⁠⁠‌⁠​​‌‍‍‌‌‍​⁠​‍​‍​‍⁠​​‍​‍‌‍‍‌‌‍‌​​‍​‍​⁠‍‍​‍​‍‌‍⁠‍‌‍‌‌‌⁠‌⁠​‍​‍​‍⁠​​‍​‍‌‍‌​​‍​‍​⁠‍‍​‍​‍​⁠​‍​⁠​​​⁠​‍​⁠‌‌​⁠​‌​⁠​​​⁠​‍​⁠‌​​‍​‍​‍⁠​​‍​‍‌‍‍​​‍​‍​⁠‍‍​‍​‍​⁠​‌‌⁠‌‍‌​‌‍‌​⁠⁠‌​⁠‍‌‍​‍‌​‌​‌​‍‍‌​⁠⁠‌‍​‍‌‍‍⁠‌‍‍⁠‌‍‌‍‌​⁠⁠‌​‍‍‌‌​‍​‍​‍‌⁠⁠‌

But “categories on four 90-second clips, and I’d value input on the learning outcomes and” — that’s a lot for 90 minutes; I’d calibrate on one about 45s clip, then code 2 clips with a ±150 ms tolerance for boundaries/holds and simple 0/1 per unit for gaze/mouth. Score per unit and target ≥80% agreement or kappa ≥.70 using an ELAN CSV export — does that fit your group’s level?

‌⁠‍⁠​‍​‍‌⁠‌​​‍​‍​⁠‍‍​‍​‍‌⁠​⁠‌‍‍‌‌‍‌⁠‌‍⁠‍‌‍⁠​‌‍​‌‌‍⁠‍‌‍‌⁠‌⁠‌‌‌‍​‌‌‍‌⁠‌‍‌‌‌‍‍‌‌‍⁠‍‌⁠‌​‌‍‌‌‌⁠​‍‌⁠​​‌⁠​‍‌‍‌‌‌⁠‌​‌‍‌‌‌⁠​‍​‍​‍​‍⁠​​‍​‍‌‍‍⁠​‍​‍​⁠‍‍​‍​‍‌⁠​‍‌‍‌‌‌⁠​​‌‍⁠​‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍‌‌‍‌​​‍​‍​⁠‍‍​⁠‌‍​⁠‌⁠​‍⁠​​‍​‍‌‍‌​​‍​‍​⁠‍‍​‍​‍​⁠​‍​⁠​​​⁠​‍​⁠‌‌​⁠​‌​⁠​​​⁠​‍​⁠‌⁠​‍​‍​‍⁠​​‍​‍‌‍‍​​‍​‍​⁠‍‍​‍​‍‌‌​​​⁠‍‌​⁠‌‍‌​​‌‌‍‍‌‌⁠​‌‌​​⁠‌⁠‌​‌⁠‌​‌​‌​‌​​‌‌‌‍‌‌⁠‌‌‌‌‌‌‌‌‌​‌‌⁠⁠​‍​‍‌⁠⁠‌​

In a 90-minute CEU I ran, we loaded an ELAN template with hotkeys and a hidden gold tier; boundaries scored 1 if within ±250 ms of the reference, holds 1 if ≥500 ms, gaze shift and mouth posture as exact category matches — then we computed quick percent agreement per tier for feedback. If you keep the “four 90-second clips,” pre-seed the first 15 seconds with that gold to calibrate and then unlock the rest for independent coding in November. Caveat: ±250 ms may be tight for newer interpreters, so I sometimes relax to ±300 ms.

‌⁠‍⁠​‍​‍‌⁠‌​​‍​‍​⁠‍‍​‍​‍‌⁠​⁠‌‍‍‌‌‍‌⁠‌‍⁠‍‌‍⁠​‌‍​‌‌‍⁠‍‌‍‌⁠‌⁠‌‌‌‍​‌‌‍‌⁠‌‍‌‌‌‍‍‌‌‍⁠‍‌⁠‌​‌‍‌‌‌⁠​‍‌⁠​​‌⁠​‍‌‍‌‌‌⁠‌​‌‍‌‌‌⁠​‍​‍​‍​‍⁠​​‍​‍‌‍‍⁠​‍​‍​⁠‍‍​‍​‍‌⁠​‍‌‍‌‌‌⁠​​‌‍⁠​‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍‌‌‍‌​​‍​‍​⁠‍‍​⁠‌‍​⁠‌⁠​‍⁠​​‍​‍‌‍‌​​‍​‍​⁠‍‍​‍​‍​⁠​‍​⁠​​​⁠​‍​⁠‌‌​⁠​‌​⁠​‌​⁠​​​⁠​‌​‍​‍​‍⁠​​‍​‍‌‍‍​​‍​‍​⁠‍‍​‍​‍‌⁠​⁠‌​​⁠‌‍​⁠‌​‍​‌‍‍‍‌‍‍​‌‌​⁠‌‌‌‌‌‌​‌​⁠​‌‌⁠​‌‌‌‌‌‌​‌​​⁠​​‌​⁠‍‌⁠‍‍​‍​‍‌⁠⁠‌​

Pre-send a codebook and ELAN template; add a ‘comment’ tier… Quick pairwise kappa at end — want my template?

‌⁠‍⁠​‍​‍‌⁠‌​​‍​‍​⁠‍‍​‍​‍‌⁠​⁠‌‍‍‌‌‍‌⁠‌‍⁠‍‌‍⁠​‌‍​‌‌‍⁠‍‌‍‌⁠‌⁠‌‌‌‍​‌‌‍‌⁠‌‍‌‌‌‍‍‌‌‍⁠‍‌⁠‌​‌‍‌‌‌⁠​‍‌⁠​​‌⁠​‍‌‍‌‌‌⁠‌​‌‍‌‌‌⁠​‍​‍​‍​‍⁠​​‍​‍‌‍‍⁠​‍​‍​⁠‍‍​‍​‍‌⁠​‍‌‍‌‌‌⁠​​‌‍⁠​‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍‌‌‍‌​​‍​‍​⁠‍‍​⁠‌‍​⁠‌⁠​‍⁠​​‍​‍‌‍‌​​‍​‍​⁠‍‍​‍​‍​⁠​‍​⁠​​​⁠​‍​⁠‌‌​⁠​‌​⁠​‌​⁠​​​⁠​⁠​‍​‍​‍⁠​​‍​‍‌‍‍​​‍​‍​⁠‍‍​‍​‍‌⁠‌‍‌⁠‌‌​⁠‌‌‌⁠​‌‌⁠‌​‌‌‍‌‌‌‍‌​⁠‌‍‌‍‌‌‌​‍‍‌​⁠‌‌‍‌​​⁠​⁠‌‌‌​‌‍⁠‍​⁠​⁠​‍​‍‌⁠⁠‌​