Skip to content
e6quPublic

About

English ... do you speak it

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

emdysi

emdysi is an English parser and prose checker, written in Rust and licensed under the GNU Affero General Public License 3.0 or later. Its command-line tool, en:

  • parses English into phrase structure and meaning, and says when a sentence is ambiguous;
  • checks spelling and grammar, claiming an error only when the grammar can prove it, and applies safe fixes;
  • enforces style guides: plain language, document structure, consistent terminology, and the tells of machine-written prose.
cargo build --release
target/release/en check draft.md
draft.md:4:58: warning [core.grammar-errors] Your subject doesn't agree in number with the verb 'are'.
    delivered a robust and seamless upgrade. The new version are faster. It
                                                             ^^^
    suggestions: is

The tutorial walks through checking, fixing, parsing, rule packs, glossaries and custom rules, with real output.

How it works

emdysi runs the English Resource Grammar (ERG), a broad-coverage HPSG grammar of English, with its own Rust implementation of the DELPH-IN processing stack: a TDL reader, typed feature structures and unification, the REPP tokenizer, chart mapping, morphology, and a packing chart parser. On the ERG's gold treebanks (dated measurements in evaluation.md):

  • it parses 97% of the grammatical items (3,011 of 3,096);
  • it finds the gold analysis among its readings for 96% of the constructed test suites' items, and 63% of the sentences of a Sherlock Holmes story;
  • each analysis comes with its meaning as Minimal Recursion Semantics (MRS), identical to the ERG's gold MRS for every gold analysis checked (2,966).

A parse ranker trained only on license-clean gold trees picks the gold analysis first for 81.7% of held-out sentences. Its scores are calibrated probabilities, which lets emdysi report ambiguity: readings are grouped by meaning, and when a second meaning keeps at least 5% of the probability, the sentence is reported as ambiguous, with what differs (I saw the man with the telescope: with(saw, telescope) 50%, with(man, telescope) 40%).

Famous stress sentences get full analyses and no error claims: Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo, had had had, garden paths, center embedding (corpora/stress). Constructions and words the ERG lacks are added in emdysi's own grammar files (grammar/emdysi), such as comparative correlatives (The more you read, the more you know) and US-style dates.

Sentences the grammar rejects are parsed again with the ERG's grammar-error variant, which names the error (agreement, verb form, article, ...). An error is claimed only when every best analysis of that variant names it and a correction makes the sentence grammatical; on edited text in five genres the checker claims about 2 errors per 1,000 sentences of the development set (evaluation).

Install

Each tagged release attaches en for Linux (x86-64), macOS (Apple silicon) and Windows (x86-64). Or build it with Rust 1.85 or later:

cargo install --git https://github.com/e6qu/emdysi emdysi

The binary is self-contained: the grammar, rule packs and word lists are built into it. On first use it writes the grammar to its cache directory (below) and compiles it, which takes a few seconds; --grammar DIR or EMDYSI_GRAMMAR uses another copy. It needs about 0.6 GB of memory, and up to about 1.2 GB more for each sentence parsed at once: by default one per core, as far as the memory available allows (--threads N to choose). Local models are optional (cargo install with --features llama for GGUF files; any OpenAI-compatible server works without it).

Usage

en check draft.md
en check --format markdown notes.txt
en fix draft.md > fixed.md
en parse --derivations --mrs essay.md
en check --format json draft.md      # or sarif, for CI
en lsp                               # a language server, for editors
en mcp                               # tools for agents (Model Context Protocol)
en packs

Input is plain text or Markdown (chosen from the file extension, or with --input). Reports are plain text or Markdown for people, and JSON or SARIF for programs (--format):

  • --format json prints one object per file: each diagnostic with its rule, severity, message, byte offsets, lines and columns (from 1, columns in characters, the end exclusive), the text found, and the replacement en fix applies (fix) or other candidates (suggestions). en parse --format json gives each sentence's analysis: whether it parsed with a full grammatical reading, its tree, and how it is ambiguous.
  • --format sarif prints one SARIF 2.1.0 log for all files, with fixes, for code review and CI tools (GitHub code scanning, for one).

--fail-on warning makes the exit status 1 when a warning or error is found (grammar errors the grammar can prove are warnings). en --help lists every option.

In an editor

en lsp is a language server: diagnostics appear as you write (after a pause in typing, and on open and save), and each fix or suggestion is a quick fix. When a document is first opened, the findings that need no parse (spelling, words and phrases) appear at once and the rest when its sentences are parsed. The grammar is loaded once and parses are cached, so only the sentences you change are parsed again. Options after lsp (--pack, --glossary, --min-severity warning, ...) apply to every document.

In VS Code, install the emdysi extension (emdysi.vsix, attached to each release) and en: the VS Code guide covers installing both on each system, what you see and how to fix it, settings (packs, glossaries, rules to disable, the least severity shown), Workspace Trust and troubleshooting. In Neovim (0.11):

vim.lsp.config('emdysi', { cmd = { 'en', 'lsp' }, filetypes = { 'markdown', 'text' } })
vim.lsp.enable('emdysi')

In Helix (languages.toml):

[language-server.emdysi]
command = "en"
args = ["lsp"]

[[language]]
name = "markdown"
language-servers = ["marksman", "emdysi"]

Any editor with a generic language-server client works the same way.

For agents

en mcp serves check, fix and parse as tools over the Model Context Protocol (stdio), with the same JSON as --format json. The grammar is loaded once, on the first call (about 5 s), and parses are cached, so checking a draft again after an edit takes milliseconds for the unchanged sentences. With Claude Code:

claude mcp add emdysi -- en mcp

or in any client's configuration:

{"mcpServers": {"emdysi": {"command": "en", "args": ["mcp"]}}}

Options after mcp (--pack, --glossary, --disable, --model, ...) set the defaults for every call; a call can name its own packs, disable and min_severity. Without MCP, en check --format json --min-severity warning gives an agent the findings that are worth acting on.

The default rule packs are:

  • core: spelling (with safe automatic fixes), grammatical errors named by the grammar, US/GB spelling consistency, a/an, repeated words. Every rule here claims an error, so each reports only what it can show.
  • ai-tells: vocabulary, stock phrases and constructions over-represented in machine-written prose (contrast frames, also across two sentences; setups such as here's the thing; rhetorical questions answered at once; runs of one-line paragraphs; closing questions to the reader; mechanical openers such as Moreover; both-sides framing; trailing participial clauses, three-part lists, em-dash density, chat-assistant residue), and, for a whole document, how densely these tells occur.
  • plain-style: passive voice, long sentences, intensifiers, wordy phrases, expletive there, tense shifts.
  • substance: hedged, vague and unsupported claims (seems to, studies show without a citation, a number of, clearly), and sentences and phrases that add nothing: filler (needless to say), platitudes (Only time will tell.), sentences made only of abstractions (The future belongs to those who embrace change.) and maxims (Clarity beats complexity.), the last two found in the grammar's semantics.
  • structure: the main point first, a clean heading hierarchy, short paragraphs, specific and parallel headings and list items, procedures as numbered instructions.
  • terms: acronyms defined at first use, one spelling per term, glossary terms instead of deprecated ones (--glossary FILE: TOML, TBX or a Vale vocabulary), no coined words, hyphen chains or noun stacks.

Opt-in packs (--pack NAME): microsoft, google and elastic (word choice from those style guides, via their Vale packages), wordlists (hedges, weasel words, fillers), equality (insensitive wording, from the data behind alex), languagetool (LanguageTool's English rules that need no tagger or whose parts of speech and chunks the parse gives, 3,418 of them), gitlab (GitLab's documentation style rules), slop (bheijden/slop's patterns of machine-written prose), proselint (proselint's usage advice), coverage (sentences the grammar could not analyse fully, and ambiguous ones) and decisions (checks that ask a local model). Write your own packs in TOML: see rules.md.

Optional local models, never bundled: en rewrite proposes rewrites of sentences the rules cannot fix, kept only if the grammar accepts them and their meaning holds, and en decide asks a model a question with fixed answers. They work with a GGUF file through llama.cpp (a build with --features llama) or any OpenAI-compatible server (llama-server, mlx_lm.server, Ollama, LM Studio). See models.md.

The first run compiles the grammar (a few seconds) and caches it in $EMDYSI_CACHE_DIR, $XDG_CACHE_HOME/emdysi or ~/.cache/emdysi, never in the repository; later runs load it in under a second. Parsing takes from tens of milliseconds to a few seconds per sentence, so a long document takes minutes the first time; parses are cached (en-parses in the same directory, at most EMDYSI_CACHE_MB megabytes, default 256), so checking it again after an edit parses only the sentences that changed (this README: 71 s, then 2.8 s unchanged). --no-cache or EMDYSI_NO_CACHE turns the cache off.

Roadmap

The plan, with where the work stands, its priorities and how to pick it up, is in docs/plan.md. In short:

  1. Speed of a first check. Parsing takes about 1.3 s a sentence, so a long document takes minutes the first time (rechecks parse only what changed). What is left is a faster parser core.
  2. Grammar errors without a model. 60 of 197 real grammar errors from documentation histories are caught (spelling: 167 of 177); most of the rest parse as correct English.
  3. A small model with the grammar (M20). Measured with Gemma 4 E2B: it chooses well among the corrections the grammar verifies, but neither it nor models up to 12B locate errors well enough to add much (12 of the 34 errors where a perfect chooser gains, against 7 without a model).
  4. Coverage of edited text (M19) and machine-written prose from current models (M21).

Documentation

Document Contents
docs/tutorial.md A walk through the tool, with real output
docs/rules.md Rule packs, every rule kind, glossaries
docs/models.md Optional local models: backends, decisions, rewriting
docs/evaluation.md Measured accuracy: gold treebanks, edited text, real errors, minimal pairs
docs/decisions.md The design decisions and why
docs/plan.md Milestones and status
docs/prior-art.md Survey of existing tools and data
docs/vendored.md Every vendored resource, with its source, version and license
docs/dependencies.md Rust dependencies and their verified licenses
CONTRIBUTING.md Building, testing, measuring and extending emdysi

Layout

Path Contents
crates/emdysi-tdl Reader for TDL, the grammar-definition language of the ERG
crates/emdysi-hpsg Type hierarchy, typed feature structures, unification, morphology, chart parser, MRS
crates/emdysi-repp REPP tokenizer with character offsets
crates/emdysi-parse The pipeline: tokenizing, tagging, token mapping, lexical lookup, parsing, ranking, ambiguity
crates/emdysi-text Markdown and plain-text blocks with source offsets; sentence segmentation
crates/emdysi-check Document analysis, rule packs, diagnostics, fixes, reports
crates/emdysi-lm Optional local language models
crates/emdysi-rewrite Guarded rewriting and typo correction
crates/emdysi The command-line tool, installed as en
grammar/erg The vendored English Resource Grammar (MIT), unchanged
grammar/emdysi emdysi's extensions of the ERG
packs/ Built-in rule packs
data/ Word lists (SCOWL, cspell), ERG error texts, rule data from other linters
corpora/ Test and evaluation corpora, each with its license
editors/vscode The VS Code extension (guide)
scripts/ The vendoring check, the rule importer, the tiny test model

License

emdysi is free software: you can redistribute it and modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, version 3 of the License or (at your option) any later version (LICENSE). It is distributed in the hope that it will be useful, but without any warranty. If you run a modified version as a network service, the AGPL obliges you to offer its source to the service's users. emdysi was MIT-licensed until 2026-10-17 (decision D15 in docs/decisions.md). It includes third-party material under its own licenses, all compatible with the AGPL: see THIRD_PARTY_NOTICES.md and docs/vendored.md.

About

English ... do you speak it

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages