ProfiLLM brings agentic LLM profiling to ride-hailing dispatch
ProfiLLM is an agentic LLM pipeline that generates utility-aligned driver behavior profiles for production ride-hailing dispatch, achieving measurable GMV and completion rate gains when deployed on DiDi's live dispatcher.
Score breakdown
ProfiLLM demonstrates that an agentic LLM pipeline can move beyond structured numerical features in a live, millisecond-latency industrial dispatcher and produce measurable improvements in real-world GMV and completion rates — validated by a 14-day online A/B test on DiDi's production system.
- 01ProfiLLM is an agentic LLM pipeline for utility-aligned user profiling in production ride-hailing dispatch.
- 02It addresses three constraints: context window limits, long-tail users with sparse interactions, and profiles that don't improve downstream utility.
- 03Module 1 (Tool-Augmented Global Knowledge Mining) equips an LLM agent with 27 analytical tools to mine platform-scale data.
ProfiLLM tackles the problem of integrating LLMs into industrial ride-hailing dispatch as semantic feature extractors over platform-scale behavioral logs — a space the paper describes as compelling but under-explored. Production matching pipelines are dominated by structured numerical features, yet contextual behavioral signals (e.g., a driver's habitual aversion to certain regions) are naturally expressible as LLM-generated profiles. Scaling this to a live, millisecond-latency dispatcher requires solving three intertwined constraints simultaneously: log volumes that exceed any LLM's context window by orders of magnitude, a long-tail user distribution where most drivers have too few interactions for per-user profiling, and the challenge that surface-fluent profiles do not necessarily improve downstream prediction utility.
A 14-day online A/B test confirmed consistent real-world gains: +0.47% GMV, +0.33% Completion Rate, and -0.82% Cancel-Before-Accept rate.
The system is composed of two modules. The first, Tool-Augmented Global Knowledge Mining, equips an LLM agent with 27 analytical tools to mine platform-scale data, producing reusable global knowledge, adaptive user clustering rules, and region-level supply-demand priors. The second, Utility-Aligned Profile Exploration, generates multiple candidate profiles per cluster, evaluates them through a lightweight downstream utility proxy, iteratively refines the best candidates, and constructs preference pairs for DPO fine-tuning.
Deployed on DiDi's production dispatcher, ProfiLLM achieved up to +6.14% relative AUC improvement in outcome prediction and up to +4.35% GMV gain in dispatching simulation. A 14-day online A/B test confirmed consistent real-world gains: +0.47% GMV, +0.33% Completion Rate, and -0.82% Cancel-Before-Accept rate.
Key facts
- 01ProfiLLM is an agentic LLM pipeline for utility-aligned user profiling in production ride-hailing dispatch.
- 02It addresses three constraints: context window limits, long-tail users with sparse interactions, and profiles that don't improve downstream utility.
- 03Module 1 (Tool-Augmented Global Knowledge Mining) equips an LLM agent with 27 analytical tools to mine platform-scale data.
- 04Module 2 (Utility-Aligned Profile Exploration) generates candidate profiles per cluster, evaluates them via a utility proxy, and uses DPO fine-tuning.
- 05Deployed on DiDi's production dispatcher, it achieved up to +6.14% relative AUC improvement in outcome prediction.
- 06Dispatching simulation showed up to +4.35% GMV gain.
- 07A 14-day online A/B test yielded +0.47% GMV, +0.33% Completion Rate, and -0.82% Cancel-Before-Accept rate.
Topics
Summary and scoring are generated automatically from the original article. We always link back to the publisher and never republish images or paywalled content. Last processed Jun 18, 2026 · 10:40 UTC. How this works →