Sembrelio

Placeholder text. The final wording will follow.

How we measure

This page describes the method of the assessment in short. Details are documented in the thesis.

Which AI services

Questions are sent through the official APIs of OpenAI (with web search) and Google Gemini (with Google Search grounding). This is an approximation of AI services, not a replica of consumer apps. Answers in those apps can differ.

Which questions

Each assessment uses 20 questions. 10 contain the name of the business, for example which services it offers or where it operates. 10 do not contain the name and ask, as a potential customer would, for providers in a category and location. Questions are asked in German by default. A person reviews the questions before they are asked.

How answers are evaluated

Statements about the business are compared with the facts from the questionnaire. Every figure is shown with its count, not only as a percentage:

  • Factual accuracy: share of checked statements that are correct
  • Coverage: share of expected facts that are stated correctly
  • Visibility: share of answers naming at least one provider that name the business
  • Other providers named and sources cited

Figures based on fewer than 5 answers are marked as limited data.

AI use and human review

Statements are extracted and checked with an AI model (Anthropic Claude). A person reviews every report before it is released. Reports are labelled as AI-generated.

Limitations

  • AI answers change over time. The search results behind the services cannot be fixed.
  • The location is stated only in the question text.
  • Results of the free assessment are preliminary because they are compared with self-reported facts.
  • An assessment makes no statement about future visibility, revenue or customer numbers.