vLLM on Ascend, deployed to ModelArts

Upstream tutorials assume you know every node's IP when you write the script.
ModelArts doesn't work that way: it allocates nodes at deploy time, publishes
their addresses in a rank table, mounts your folder at /scripts and the
weights at /model, and runs the same command on every node. This skill is
that gap.

Two modes

Generate — build a deploy folder from an upstream tutorial page. The
procedure below.

Review — audit a folder that already exists, generated by this skill or
not, and propose changes. See references/review.md. Report findings by
severity and stop; the user decides what to apply.

Generate

1. Find and fetch the upstream page. Pages live under
   https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/<Page>.html,
   but <Page> is the page name, not the model name, and the two often differ
   — a family is split across pages (Qwen3-Dense.html covers Qwen3-32B;
   Qwen3-235B-A22B.html is separate). Guessing the URL from the model name
   404s more often than not.
   To find the right page, list the upstream repo directory —
   https://api.github.com/repos/vllm-project/vllm-ascend/contents/docs/source/tutorials/models
   — and match the model to a filename. That API is rate-limited for
   unauthenticated callers and may return a rate-limit JSON instead of a
   listing; when it does, probe candidate names directly —
   curl -o /dev/null -w '%{http_code}' https://raw.githubusercontent.com/vllm-project/vllm-ascend/main/docs/source/tutorials/models/<Name>.md
   — trying hyphen and no-hyphen forms (GLM5.3 is a page, GLM-5.3 is not).
   The rendered models index is a stub with no links; it will not help. Confirm
   with an HTTP 200 before reading.
   Read the headings out of the actual HTML — a summarizing fetch will invent
   plausible section structure that isn't there. The subsections of "Online
   Service Deployment" are the available topologies. Never invent one the page
   doesn't document.
   Check for tabbed variants (A2 / A3 / dspark) and read section 4 for
   image tags — both are per-platform and easy to miss. See
   references/output-shape.md.
2. Enumerate the topologies, then ask which to generate. List every one the
   page documents with its node count, shape, and trade-off, and let the user
   pick — don't choose silently. List every platform the page covers, not
   just A2/A3: a 300I DUO or 310P tab is a real option and the user may hold
   that hardware. Never drop one because it is unfamiliar — say what it needs
   and let them choose. Flag any that the page names in prose but gives
   no code block for: those can only be scaffolded, not completed.
3. Confirm the target with the user: platform, node count, ModelArts
   inference version (1.0 or 2.0). Don't guess these.
   On 2.0 also settle the 部署单元 split: multi-node takes two units, the
   first holding one 资源实例 and the second the rest, so group 0 is exactly one
   node; single-node is one unit of one 资源实例. See
   references/modelarts-spine.md.
   A2 is 8 devices/node, A3 is 16 — one A3, 8 NPUs of two chips each, which
   pages write as either 128GB × 8 or 64GB × 16. Multiply out the page's own
   dp_local × tp and check it lands on 8 or 16. See
   references/topology-matrix.md.
4. Copy the spine into the new folder, byte-for-byte, from assets/spine/:
   start.sh always, and setup_rank_env.sh for multi-node. Don't edit the
   copies — fix the asset and re-copy, so every layout stays identical.
   setup_rank_env.sh is sourced, not executed, so its file mode does not
   matter; start.sh is the service's start command, so chmod 755 it after
   copying — setting a mode is not editing the file, and the byte-for-byte rule
   is about content.
   Single-node layouts need no rank table, dispatch, or NIC preamble — but they
   still take start.sh.
5. Write the scripts, starting from assets/templates/. Apply every rewrite
   in references/adaptation.md — that file is the actual work of this skill.
6. Write meta.yaml and README.md recording the source URL, today's date,
   the image tag from section 4, verified: false, and any gap you left.
   Prose is written in Chinese — see references/output-shape.md.

Read references/modelarts-spine.md before step 5 — the rank table and the
group-0 constraint are what the scripts are built around.

Rules

• Rewrite the wiring, not the engine. Paths, addresses, ports, environment
  and role dispatch are yours — the platform assigns those. Every engine flag,
  including everything inside --additional-config, stays as the page has it.
• Nothing you generate is verified. Say "derived from the upstream page, not
  validated on hardware." Never imply it's known-good.
• Never edit a layout's copy of the spine. Fix assets/spine/ and re-copy;
  see assets/README.md first.
• Never invent a value the page doesn't give. Leave TODO(gap): in the
  script, list it in meta.yaml, and tell the user. A plausible guess is worse
  than a marked hole.
• Group 0 is the constraint that bites. API-serving ranks inside it,
  headless ranks outside. run.sh must validate this and fail fast.
• Model path is the mount path, conventionally /model on both 1.0 and 2.0
  — never upstream's weights path, and never under /root (0700 locks out
  1.0's ma-user). Confirm it.
• Keep comments lean. Comment only what the code cannot show; rationale
  belongs in the README.
• A review reports, it does not edit. Findings first, with the proposed
  change for each; apply only what the user picks. A folder in production may
  differ from the page deliberately.

References

Author

Written by xuopoj.
Source: <https://github.com/xuopoj/ascend-delivery>

Questions, corrections, or a page this skill handles badly? Open an issue at
<https://github.com/xuopoj/ascend-delivery/issues>. Hardware findings are
especially welcome — nothing this skill generates is verified until someone
runs it on real NPUs.