How does Story Check work?

The purpose of Story Check is to combine the newsroom’s continuously updated standards of producing accurate texts with the advanced language capabilities of selected LLMs, providing reliable and thorough spell checking.

Story Check works through a browser extension available for Chrome, Edge, and Firefox. When the Story Check extension is installed, the user must grant it permission to access the relevant website and text fields. Once access has been granted, Story Check can extract the text you want to proofread and open a pop-up displaying the suggested changes.

Story Check distinguishes between regular text that should be processed by the spell checker and non-text elements, such as HTML, embeds, and other technical content, which are excluded from the spell-checking process. Styling and formatting are preserved throughout.

The extracted text is transferred to Ritzau’s systems for processing and stored in our logs.

Different instructions are necessary for different purposes. That means that Story check needs to be flexible regarding the choice of language rules. A good example is that the rules for a header is different from a body text. In a header, you would usually like the spell checker to remove the full stop at the end, while you would like it to suggest a missing full stop in the body tekst.

To accomodate this need Story Check works with “language profiles” that operates as containers of sets of language rules. An editor with manager role will be able to bundle rules in a profile, that for example serves the purpose of spell checking the header. In the CMS you chose this profile to handle the header field in the CMS.

The context-specific rules are selected using RAG.

Each language rule includes an automatically generated retrieval text that describes the rule in a standardized form. The RAG system compares the text being checked with these retrieval texts and assigns a relevance score to each rule. Rules that exceed a defined relevance threshold are then included in the prompt.

This ensures that the prompt only contains rules that are relevant to the specific text. For example, a rule about how to spell the names of American political parties would not be included when checking an article about Norwegian football, where it would only introduce unnecessary noise.

Story Check offers two levels of spell checking: a fast version and a thorough version (see step 6). If you choose the thorough spell check, your text is also checked for the correct spelling of named entities, such as people, places an organizations.

This is done through an AI-based analysis of the text that identifies relevant entities. These entities are then compared with open data sources to verify that they are spelled correctly in the specific context in which they appear.

For example, if a politician named Pär Ohlsson is misspelled as “Olsson” – which may also be the correct surname of a football player – Story Check uses the surrounding context to identify the intended person and suggest the correct spelling.

Our language editor in the news room is responsible for maintaining and updating the language rules in Story Check. The editor updates the instructions associated with these rules in the following situations:

1. When official language standards are changed by national language authorities.

2. When newsroom management decides to change or introduce new language policies.

3. When language editors receive relevant feedback from newsroom journalists through the feedback tool in Story Check.

The language editor is also responsible for maintaining the entity data used by Story Check. That could be in situations when new names appear or existing names change and the relevant information has not yet been updated in the open data sources that are used by Story Check.

When a user activates Story Check on a text, they can choose between two spell-checking modes: Fast and Thorough.

The main difference between the two modes is the type of LLM used and the level of analysis performed.

The fast option uses a lighter and faster LLM that is well suited for identifying common spelling and language errors, but is less reliable in complex or ambiguous cases. This option is particularly useful in situations where speed is critical, such as live blogging, breaking news, and front-page updates.
The thorough option uses a more capable LLM and performs a more detailed analysis of the text and the applicable language rules. It also includes entity spell checking (see step 5), which verifies names, places, organizations, and other named entities. This additional processing makes the Thorough option slower than the Fast option.

In addition to the primary LLM used for each mode, Story Check also has a separate fallback LLM for both the Fast and Thorough options. The fallback models are used if the primary model fails to return a valid response or experiences unusually high latency.

When the text is returned from the LLM, it is compared with the original text, and the differences are presented to the user as suggested corrections.

However, the LLM output is not the only source of corrections. In parallel with the LLM-based spell check, Story Check also checks the text against a list of forced corrections. These work like predefined search-and-replace rules and ensure that certain words or names are always written in a specific way. For example, a newsroom may decide to use a particular spelling of a company name that is not reflected in the open entity data.

Story Check also checks the text against a list of Disallowed words. This list contains words or names that are not necessarily misspelled, but should only be used in specific contexts, such as in quotations. Examples may include slang, offensive expressions, or other terms that require special attention.

Finally, any entity corrections identified through the entity-checking process described in step 4 are added to the final set of suggestions.

The final set of suggestions is displayed in the browser extension pop-up, with proposed corrections highlighted in red in the original text. The suggestions are also listed in the panel on the right, where each correction can be accepted individually or multiple corrections can be accepted at once.

Once the user has reviewed and managed the suggested corrections, the approved version of the text can replace the original by clicking “Accept”. The pop-up then closes automatically.

HTML, embeds, and other non-text elements remain unchanged throughout the process.