{
  "generator": {
    "name": "Symvanta",
    "url": "https://symvanta.com"
  },
  "page": {
    "url": "https://symvanta.com/architecture/vllm",
    "title": "vLLM Architecture: How It Actually Works",
    "description": "How the vLLM codebase is structured: module map, load-bearing symbols, and dependency structure, generated by Symvanta.",
    "datePublished": "2026-08-11",
    "dateModified": "2026-08-11"
  },
  "repository": {
    "name": "vllm-project/vllm",
    "url": "https://github.com/vllm-project/vllm",
    "commit": "63ac04a",
    "license": "Apache-2.0"
  },
  "graph": {
    "modules": 500,
    "dependencyCycles": 0,
    "modularityQ": 0.85,
    "largestCycleFiles": null,
    "mutuallyRecursiveGroups": 13
  },
  "summaryMarkdown": "vLLM is an inference engine for large language models: a continuous-batching\nscheduler feeds a paged KV cache and tensor-parallel model execution, behind\nan OpenAI-compatible HTTP server. Symvanta's graph at `63ac04a` detects 500\nfunctional modules at modularity Q=0.85, and the biggest clusters are the\nplumbing every model file imports: the logger and platform interface at 4130\nsymbols and the tensor-parallel state accessors at 3540.\n\nThat shape is the structural signal. Hundreds of model implementations and\nkernels all call the same logging, platform, and distributed-state helpers,\nso the utility clusters grow huge while the engine itself stays compact: the\nV1 scheduler cluster holds 1435 symbols and the engine core client 1032. The\ngraph found no module-level dependency cycles across the 500 modules.",
  "subsystems": [
    {
      "name": "Logger and platform interface",
      "descriptionMarkdown": "`_VllmLogger.warning_once` from `vllm/logger.py` sits with the `Platform`\ninterface (`is_rocm`, `is_cuda`, `fp8_dtype`, `get_device_capability`) in one\n4130-symbol cluster, the largest in the repo. Every model implementation\nlogs once and asks what hardware it is on, and those two habits pull the\nwhole model zoo toward this cluster."
    },
    {
      "name": "Tensor parallel state",
      "descriptionMarkdown": "`get_tp_group` and the accessors around it in\n`vllm/distributed/parallel_state.py`, plus `init_logger` and\n`default_weight_loader`. 3540 symbols. Its 404 calls into the logger cluster\nand the 377 coming back are the heaviest edge pair on the map."
    },
    {
      "name": "Multimodal processing info",
      "descriptionMarkdown": "`BaseProcessingInfo` (`vllm/multimodal/processing/context.py`), the\n`TokenizerLike` protocol, and `InputProcessingContext`: the interface every\nmodel uses to declare its HF config, processor, and tokenizer. 1790 symbols."
    },
    {
      "name": "LLM entrypoint and test harness",
      "descriptionMarkdown": "`LLM.generate` from `vllm/entrypoints/llm.py` clusters with the test\nhelpers that drive it (`multi_gpu_test`, `check_logprobs_close`,\n`create_new_process_for_each_test` from `tests/utils.py`). 1683 symbols, and\nits hub ranks in the PageRank top 10 because the test suite calls\n`LLM.generate` from everywhere."
    },
    {
      "name": "V1 scheduler and config factories",
      "descriptionMarkdown": "The engine's brain: `Scheduler.schedule` and `Scheduler.add_request` from\n`vllm/v1/core/sched/scheduler.py`, `Request.num_tokens`, KV cache spec\nregistration, and the `create_vllm_config` / `create_scheduler` test\nfactories. 1435 symbols. The canonical flow below starts here."
    },
    {
      "name": "Engine core client",
      "descriptionMarkdown": "`AsyncMPClient` from `vllm/v1/engine/core_client.py`, the msgpack encoder,\nand `AsyncLLM.generate`: the async client side of the engine core process\nboundary. 1032 symbols."
    },
    {
      "name": "Shared infrastructure",
      "descriptionMarkdown": "The rest of the top 10: GroupCoordinator ranks (1884 symbols, hub\n`GroupCoordinator.is_first_rank`), SamplingParams and serving utils (1522,\nhub `random_uuid`), EngineArgs and CLI parsing (1046, hub\n`LLM.get_default_sampling_params`), and Quantization config and debug\nlogging (954, hub `_VllmLogger.debug_once`)."
    }
  ],
  "startReading": {
    "symbols": [
      {
        "name": "get_tp_group"
      },
      {
        "name": "AutoWeightsLoader.load_weights"
      },
      {
        "name": "LLM.generate"
      },
      {
        "name": "GroupCoordinator.is_first_rank"
      },
      {
        "name": "MultiModalRegistry.register_processor"
      }
    ],
    "endpoints": [
      {
        "method": "GET",
        "path": "/health"
      },
      {
        "method": "GET",
        "path": "/version"
      },
      {
        "method": "POST",
        "path": "/tokenize"
      },
      {
        "method": "POST",
        "path": "/detokenize"
      },
      {
        "method": "GET",
        "path": "/v1/models"
      },
      {
        "method": "POST",
        "path": "/v1/chat/completions"
      },
      {
        "method": "POST",
        "path": "/v1/completions"
      },
      {
        "method": "POST",
        "path": "/v1/embeddings"
      },
      {
        "method": "POST",
        "path": "/pooling"
      },
      {
        "method": "POST",
        "path": "/score"
      }
    ]
  },
  "requestFlow": [
    {
      "position": 1,
      "symbol": "EngineCore.step (vllm/v1/engine/core.py:580)",
      "sourceUrl": null,
      "markdown": "`EngineCore.step (vllm/v1/engine/core.py:580)`"
    },
    {
      "position": 2,
      "symbol": "Scheduler.schedule (vllm/v1/core/sched/scheduler.py:438)",
      "sourceUrl": null,
      "markdown": "`Scheduler.schedule (vllm/v1/core/sched/scheduler.py:438)`"
    },
    {
      "position": 3,
      "symbol": "KVCacheManager.allocate_slots",
      "sourceUrl": null,
      "markdown": "`KVCacheManager.allocate_slots`"
    },
    {
      "position": 4,
      "symbol": "Executor.execute_model",
      "sourceUrl": null,
      "markdown": "`Executor.execute_model`"
    },
    {
      "position": 5,
      "symbol": "Scheduler.get_grammar_bitmask",
      "sourceUrl": null,
      "markdown": "`Scheduler.get_grammar_bitmask`"
    },
    {
      "position": 6,
      "symbol": "Executor.sample_tokens",
      "sourceUrl": null,
      "markdown": "`Executor.sample_tokens`"
    },
    {
      "position": 7,
      "symbol": "Scheduler.update_from_output",
      "sourceUrl": null,
      "markdown": "`Scheduler.update_from_output`"
    }
  ],
  "diagrams": [
    {
      "url": "https://symvanta.com/architecture/vllm/module-map.svg",
      "encodingFormat": "image/svg+xml"
    }
  ]
}