Assessment rules and limitations

How GraphSec decides whether a Microsoft 365 control passed, needs attention or could not be tested.

This page explains the catalog in practical language for Microsoft 365 administrators and consultants.

View plans
01 · Assessment scope

GraphSec reviews specific settings and states in a Microsoft 365 tenant.

Each control answers one administrative question, such as whether privileged accounts are covered by an effective Conditional Access policy, whether anonymous SharePoint links are allowed or whether privileged roles have accountable owners.

The result applies to the time of the scan and the data that could be read. Missing licences, permissions or APIs are reported as testing gaps, not as passed controls.

02 · Catalog structure

The 53 controls follow the main work areas of a Microsoft 365 administrator.

The catalog covers identities, apps and OAuth, users and guests, devices, Exchange and collaboration, data governance, detection and recovery.

Control IDA stable identifier such as IAM-01.
Assessment questionThe exact setting or state to review.
Data sourceGraph endpoint, Exchange command, Teams data or manual evidence.
Decision ruleThe values that result in passed, warning or failed.
EvidenceThe technical values supporting the result.
RecommendationThe next practical administrative action.
03 · Assessment process

Every control follows the same sequence.

  1. 1
    Check prerequisites

    Confirm applicability, licensing, module availability and read permissions.

  2. 2
    Read data

    Collect the required values from the defined source.

  3. 3
    Apply the rule

    Compare the values with the documented expected and deviation criteria.

  4. 4
    Document the result

    Store status, reason, evidence, recommendation and testing limitations.

04 · Status values

The status describes the technical result.

Passed

The defined criterion is met.

Warning

An exception or incomplete coverage requires review.

Failed

The technical security criterion is not met.

Manual evidence

The tenant alone cannot provide enough evidence.

Permission missing

A required read permission was unavailable.

Licence missing

The required Microsoft feature was not licensed.

05 · Score and coverage

The score evaluates results. Coverage shows how much could be assessed.

Passed counts as 1, Warning as 0.5 and Failed as 0. Controls without sufficient data are not counted positively.

Technical score100 × (Passed + 0.5 × Warnings) ÷ (Passed + Warnings + Failed)
Test coverage100 × evaluated controls ÷ applicable controls
A high score with low coverage is not a strong overall result.
06 · Evidence

The report shows why a control received its status.

Evidence may include policies, exclusions, role assignments, sharing settings or device state. Restore tests, contracts and management approval must be supplied separately.

07 · BSI, BSIG and NIS2 mapping

Mapping explains relevance; it does not prove full compliance.

A technical result can support areas such as access control, logging, supply-chain security or recovery. Full compliance also requires governance, contracts, processes and effectiveness evidence.

08 · Limitations and disclaimer

The user remains responsible for validation, prioritisation and customer communication.

  • Microsoft APIs and licensing can change.
  • Read permissions must follow least privilege.
  • A configured policy does not automatically prove organisational effectiveness.
  • On-premises systems and unconnected services remain outside the scan.
  • The report is not legal advice, a certification or a complete compliance assessment.

Read the full disclaimer

Ready to test the workflow in your own tenant?

The installation guide explains prerequisites, authentication and a safe first run.

View installation