Context Wars: The Relevance Strikes Back

Adaptive Context Management for LLM-Based Self-Adaptive Systems

Pranav Swarup Kumar · 2025121011

---

What This Is (Ultra Concise Summary of The Report)

This repo contains my exploratory research into OS-inspired context management for LLM-based systems, submitted to Prof. Karthik Vaidhyanathan (SERC, IIIT Hyderabad). The work started from a simple frustration I shared with Prof. Karthik. That is, long LLM conversations get slow because every message ships the entire history. This idea evolved into a proposed Context Balancer module for self-adaptive architectures. I built a three-tier memory hierarchy (Hot/Warm/Cold) with semantic topic-shift detection, hit a wall with threshold fragility, pivoted to a learned MLP classifier for relevance filtering (95% precision after synthetic data generation fixed a total data-scarcity failure), and then connected everything to the MSE-K framework by implementing and evaluating the Context Balancer on a SWIM-like simulator. The fun result was that recency-weighted filtering achieves 77% token reduction with 100% decision agreement against the full-history baseline — meaning three-quarters of conversation history is provably unnecessary for adaptation decisions, as long as temporal locality is preserved alongside semantic similarity.

Report

The full writeup is at the root of this repo:

SA4S_Report_Pranav_Swarup.pdf

Running the Learned Relevance and Paging Experiment (not needed for mse-k experiment)

# create a venv
python3 -m venv venv

source venv/bin/activate

pip install numpy scikit-learn sentence-transformers matplotlib


#Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

# Pull the model used in experiments
ollama pull llama3.2:3b

# Verify it's running
ollama list

# now just run the files normally to get json results in the results directory.

The MSE-K Experiment

cd (root of the project repo)

# Run all 5 configurations (baseline + 4 balancer variants)
python run_msek_experiment.py

# Verbose mode (per-step adaptation decisions)
python run_msek_experiment.py --verbose

No external APIs, no GPU, no model downloads needed. The experiment runs in under a second.

Files:

File	What it does
swim_sim.py	SWIM-inspired web server simulator (arrival rates, servers, dimmer, utility)
context_balancer.py	The proposed Context Balancer module with cosine and recency-weighted modes
msek_loop.py	MSE-K adaptation loop with trend-aware rule-based policy
run_msek_experiment.py	Runs 5 configurations, prints comparison tables and analysis

---

Disclaimer: All code in this repository was developed with assistance from Claude Code (Anthropic). The research direction, experimental design, interpretation of results, and writeup framing are my own.

Key References

Papers that directly shaped this work:

• Donakanti, R., Jain, P., Kulkarni, S., Vaidhyanathan, K. — Reimagining Self-Adaptation in the Age of Large Language Models (ICSA 2024). The MSE-K framework and its explicit mention of the long-context gap motivated the Context Balancer.
• Parthasarathy, K., Vaidhyanathan, K., et al. — Engineering LLM Powered Multi-Agent Framework for Autonomous CloudOps (CAIN 2025). Multi-agent context sharing as a scaling challenge.
• Akshathala, S., et al. — Beyond Task Completion: An Assessment Framework for Evaluating Agentic AI Systems (arXiv, Dec 2025). The Memory evaluation pillar connects to relevance filtering metrics.
• Kulkarni, S., Marda, A., Vaidhyanathan, K. — Towards Self-Adaptive ML-Enabled Systems Through QoS-Aware Model Switching (ASE 2023). The Model Balancer concept that the Context Balancer is analogous to.
• Marda, A., Kulkarni, S., Vaidhyanathan, K. — SWITCH: An Exemplar for Evaluating Self-Adaptive ML-Enabled Systems (SEAMS 2024).
• Vaidhyanathan, K., Muccini, H. — Software Architecture in the Age of Agentic AI (ECSA 2025).
• Packer, C., et al. — MemGPT: Towards LLMs as Operating Systems (2023). OS-inspired virtual context management for LLMs.
• Moreno, G.A., et al. — SWIM: An Exemplar for Evaluation and Comparison of Self-Adaptation Approaches for Web Applications (SEAMS 2018).