How the APM (Adaptive Plasticity Module) performs against conventional training, measured on real workloads with every method run on the same GPU or CPU.
Agent tool routing. 1,000 real tools and 20,000 real user requests, with 10% of tools never seen in training. The task is to put the right tools in a short list for an AI agent.
Multilingual intent. 60 intents across 8 languages, plus adding a brand-new intent on the same day.
Exactness. Thousands of random additions and deletions, compared with a model retrained from scratch.
All runs used a state-of-the-art open embedding model. The comparison is a standard trained classifier on the same embeddings, retrained on the same hardware.
| Size | One update | Faster than retraining | Top-5 accuracy | Gap to full retrain after 1,000 updates |
|---|---|---|---|---|
| 1,000 | 1.25 ms | 993× | 93.6% | 5×10−15 |
| 2,000 | 1.37 ms | 905× | 94.8% | 3×10−14 |
| 4,000 | 2.22 ms | 557× | 95.5% | 2×10−13 |
| 8,000 | 5.57 ms | 222× | 95.5% | 1×10−12 |
| Size | One update | Faster than retraining | Top-5 accuracy |
|---|---|---|---|
| 1,000 | 30 ms | 2,346× | 93.3% |
| 2,000 | 87 ms | 811× | 94.8% |
| 4,000 | 305 ms | 233× | 95.1% |
right tool in the top 5 for tools with usage history, versus 86.3% for description search
of correct tools covered by a top-10 shortlist using just 1.1% of the catalogue's tokens
tools in the shortlist to cover 95% of requests when combined with search, versus 37 for search alone
| Language | Adaptive Plasticity Module | Trained classifier |
|---|---|---|
| English | 86.7% | 87.9% |
| German | 83.7% | 84.5% |
| Spanish | 84.4% | 84.9% |
| French | 85.1% | 84.8% |
| Hindi | 83.1% | 83.8% |
| Arabic | 77.7% | 78.5% |
| Japanese | 85.5% | 84.9% |
| Chinese | 83.8% | 84.7% |
random adds and deletes at 2.5 ms each, still equal to a fresh retrain within 1.6×10−12
to retire a tool with 219 examples, so it's never predicted again
of other predictions changed by 50 targeted corrections
to add a brand-new intent the same day, with existing accuracy kept
difference after undoing a change, back to the previous model
faster than retraining per wave as 1,000 tools arrive in 10 waves
accuracy learning 219 classes in sequence, versus 44.2% for sequential fine-tuning (full retrain: 95.8%)
to delete 2,000 records exactly
accuracy after merging models trained at 5 separate sites, without sharing data
Full benchmark notebooks are available on request. Ask for them →