AdaCall: Spending an LLM Budget on Conclusions, Not Trajectories, in Counterfactual Social Simulation
Gangyi Zhang, Qinzhe Wang, Junyi Liu, Zhuolin Li, Liangfei Jin
Team Antarctic penguins
1st Prize, Track C (Methodological and Systems Innovation), SocioVerse Challenge 2026. AdaCall is a calling layer for counterfactual experiments in SocioVerse2: it spends the LLM budget on the estimated treatment effect rather than on recomputing unchanged trajectories, and it audits both approximate reuse and the conclusion itself.
- Pay only for divergence: after an intervention, a treatment-arm agent whose prompt is byte-identical to its control-arm prompt reuses the control reply, an exact draw from the same distribution, so only diverged agents make new LLM calls and the effect estimate stays unbiased.
- Fewer calls, lower variance: on opinion diffusion (100 agents, ring and small-world networks, 20 paired replications with DeepSeek), the paired arm paid 4–54% of the calls of an independent arm and reduced the variance of the estimated effect by 3.4× to 40×.
- Cost follows spillover: 15% of post-intervention calls were paid when prompts carry no spillover (car market, one German state targeted) and 45% with group spillover (campus dining, 28% targeted).
- Audits, including a negative result: seeded audit calls flag a reuse tolerance that attenuates the effect by about 70%, but an admitted tolerance can still carry bias, so exact reuse stays the default. A placebo arm checks whether an effect comes from the message content or from being targeted.
- Drop-in for SocioVerse2: the studies use SocioVerse2's standard study format; AdaCall wraps only the LLM calling layer and leaves the platform code unchanged.
- [2026-11] 🏆 AdaCall received the 1st Prize in Track C (Methodological and Systems Innovation) of SocioVerse Challenge 2026, announced at the challenge session of LASS 2026, the 2nd Workshop on LLM Agents for Social Simulation, co-located with CIKM 2026 in Rome.
Figure 1: Overview of AdaCall. A conventional branch pays for every agent-step; with paired, divergence-triggered calls (C1) only agents whose prompt has diverged from the control arm are paid. C2 adds optional tolerant reuse with seeded audits, and C3 audits whether the conclusion itself is reproducible.
Figure 2: (left) Paid calls as a share of an independent arm, by campaign reach; (right) variance reduction from pairing with exact 95% F intervals (opinion diffusion, DeepSeek, 20 replications per network).
| Network | Reach | Paid (% of indep. arm) | Variance ratio [95% CI] | Equal-budget gain |
|---|---|---|---|---|
| Ring | 1% | 4.2% | 40.5× [16.0, 102.4] | 77.8× |
| Ring | 5% | 24.1% | 3.8× [1.5, 9.6] | 6.2× |
| Small-world | 1% | 23.0% | 8.5× [3.4, 21.6] | 13.9× |
| Small-world | 5% | 53.5% | 3.4× [1.4, 8.6] | 4.4× |
| Ring / Small-world | 100% | 87.5% | – | – |
| Testbed, target | Spillover | Replications | Paid (% of post-intervention calls) |
|---|---|---|---|
| Car market, Bavaria | none | 5 | 15.0% |
| Campus dining, 5 dorms | group | 20 | 44.5% |
| Opinion diffusion, 1–5% reach | network | 20 | 4.8–61.1% |
A global intervention (100% reach) changes every prompt, so pairing saves little; savings are largest for local interventions and sparse networks.
Requires Python 3.11 and git.
git clone https://github.com/zhanggangyi1224/AdaCall.git
cd AdaCall
bash scripts/setup.sh # clones SocioVerse2 at the pinned commit 9c3ac77, links the studies, installs it
python -m pytest adacall/tests -qpython run.py # free stand-in LLM, no API key
python run.py --mode live --study od --model deepseek-flash --reps 5 # real LLM- Stand-in mode runs every design on all three testbeds with fixed seeds and writes about 1.2 GB to
adacall/runs/. - Live mode reads the provider key from
SocioVerse2/.env(SV_LLM_API_KEY, or the provider-specific variables listed instudies/adacall_od/model.py).--studyisod,germanyorcampus. - Every reply is cached in
<out>/cache.jsonl, so an interrupted run resumes without paying twice and a rerun with the same cache makes no API calls. Each runner writessummary.jsonandruns.csvto its--outdirectory;python adacall/<runner> --helplists the options.
| Path | Contents |
|---|---|
adacall/cache.py |
Paired response cache: exact replay, paired reuse, optional numeric tolerance with audit sampling |
adacall/run_od.py |
Control, paired and independent arms on the opinion-diffusion study |
adacall/run_paired.py |
The same design on the car-market (germany) and campus-dining (campus) studies |
adacall/run_eps.py |
Tolerance sweep with audit calls (C2) |
adacall/run_audit.py |
Conclusion audit: placebo, temperature 0, paraphrase, prompt decomposition (C3) |
studies/adacall_{od,germany,campus}/ |
SocioVerse2 studies: population, environment, simulation and behaviour model |
If you find AdaCall useful in your research, please cite our work:
@misc{zhang2026adacall,
title={AdaCall: Spending an LLM Budget on Conclusions, Not Trajectories, in Counterfactual Social Simulation},
author={Zhang, Gangyi and Wang, Qinzhe and Liu, Junyi and Li, Zhuolin and Jin, Liangfei},
howpublished={SocioVerse Challenge 2026, Track C (1st Prize). Presented at LASS 2026, the 2nd Workshop on LLM Agents for Social Simulation, co-located with CIKM 2026},
year={2026},
url={https://github.com/zhanggangyi1224/AdaCall}
}We thank the SocioVerse team for the simulation platform and the SocioVerse Challenge 2026 organizers for their support.
For questions, please contact zhanggangyi1224@gmail.com.